Features, Labels, Examples, Datasets

A dataset is a contract too

A supervised dataset is not just a table. Each row is an example. Features describe what is known at prediction time. The label is the answer the model is allowed to learn from during training. The hardest part of dataset design is being honest about which columns belong in which role.

Three questions per column

What does this column mean in business terms? If you cannot say it in one sentence, the column is a future bug.
When is this value known? If it only exists after the prediction, it is leakage.
Could it accidentally encode the target? Hash-encoded categorical IDs, post-event statuses, support-call counts, and refund amounts are common offenders.

Identifiers are especially deceptive

A customer or product ID may be numeric without having meaningful magnitude. A model can memorize entities that occur in both training and validation, then fail on a new entity. Decide whether an identifier is legal together with the split strategy: rows from the same customer often belong in the same fold.

Schema as a guard

A small column-audit DataFrame is one of the highest-leverage habits in classical ML. It forces you to mark each column as feature, label, identifier, leakage risk, or post-event metadata. Once it exists, every new column has to pick a role to enter the pipeline.

Make the audit executable

Store meaning, creation time, dtype, allowed values, and leakage status in a versioned schema. Validate new data against it before training or scoring. The audit then stops being a document someone forgets and becomes the single entrance into the feature pipeline.

Code

A column-audit table for a churn dataset·python

import pandas as pd

audit = pd.DataFrame([
    {"column": "customer_id", "role": "identifier", "known_at_prediction": True, "use": False},
    {"column": "plan_tier", "role": "feature", "known_at_prediction": True, "use": True},
    {"column": "tickets_last_30d", "role": "feature", "known_at_prediction": True, "use": True},
    {"column": "days_since_signup", "role": "feature", "known_at_prediction": True, "use": True},
    {"column": "cancel_reason_text", "role": "post_event", "known_at_prediction": False, "use": False},
    {"column": "refund_amount", "role": "post_event", "known_at_prediction": False, "use": False},
    {"column": "churned", "role": "label", "known_at_prediction": False, "use": "target"},
])
feature_cols = audit.loc[audit["use"] == True, "column"].tolist()

Use the audit as the only path into the model·python

X = df[feature_cols].copy()
y = df["churned"]
assert "cancel_reason_text" not in X.columns, "leakage column slipped through"

A dataset is a contract too

Three questions per column

Identifiers are especially deceptive

Schema as a guard

Make the audit executable

Code

External links

Exercise

Progress

Comments 0