Better Data Beats Fancier Models

~28 min · data-centric, labeling, framing

Level 0Scout

0 XP0/48 lessons0/11 achievements

0/120 XP to next level120 XP to go0% complete

Why data wins

For most production problems the model architecture is not the bottleneck. The bottleneck is label quality, label coverage, feature freshness, and the gap between training data and production data. A team that spends a week on labels and a day on the model usually beats a team that does the opposite.

Five high-leverage data moves

Write a labeling guide that turns judgment calls into reproducible rules.
Sample disagreements and re-label them as a team.
Add explicit "I don't know" and "ambiguous" labels so noise has a home.
Track label quality over time the way you track model quality.
Inspect rare classes and edge cases manually before any modeling.

Andrew Ng's data-centric flip

The data-centric AI message is that for fixed model code, improving data labels often beats tuning the model. The implication is that label review is engineering work, not assistant work, and it deserves engineering time.

Code

Track inter-annotator agreement·python

from sklearn.metrics import cohen_kappa_score

kappa = cohen_kappa_score(annotator_a, annotator_b)
print(f"agreement κ={kappa:.2f}  (>0.6 acceptable, >0.8 good)")

Sample low-confidence predictions for re-labeling·python

import numpy as np

probs = model.predict_proba(X_unlabeled)
uncertainty = 1 - np.abs(probs[:, 1] - 0.5) * 2
to_review = X_unlabeled.iloc[np.argsort(-uncertainty)[:200]]

External links

Exercise

Take 100 labeled examples from your project. Have a second reviewer relabel them blind. Compute Cohen's kappa. If kappa is below 0.7, fix the labeling guide before training another model.

Progress

Progress is local-only — sign in to sync across devices.

← 🧠 What Machine Learning IsQuiz · 4 questions Next →Tabular Data Intuition

Spotted a bug or have feedback on this page?Report an Issue

Comments 0

🔔 Reply notifications (sign in)

No comments yet — be the first.