What Machine Learning Actually Is

The four-piece contract

Machine learning is not a magic box that turns data into answers. It is a contract among four pieces: a target you want to predict, the features you are allowed to use at prediction time, the data that pairs the two for training, and the metric that decides whether the model helped. Change the target and the required examples may change; change the metric and “good” may mean a different model.

Learning is not memorization

A learning algorithm changes parameters so a loss on training examples decreases. The goal is for those parameters to work on examples the model has never seen. That is not automatic. It depends on representative examples, a stable target definition, and features that truly exist at prediction time. A model that reproduces its training set and collapses on new cases has remembered answers rather than learned a useful pattern.

The model is one system component

In production, the model sits inside feature extraction, scoring, thresholding, decisions, logging, monitoring, and feedback. A model file without an input contract or rollback path is still a laboratory object. Its internal algorithm can be elegant and still be replaceable if it violates the larger system's interface.

A good score can still lie

A 0.97 ROC-AUC on a notebook split proves only that one calculation finished. Post-event refund information may have leaked into the features, or the served population may differ from the training rows. Draw the prediction timeline, verify that every feature existed before the decision, and compare the result with the simple rule or process already in use.

Write this before the first model

On one page, state whose outcome is predicted, what the outcome means, when prediction occurs, which information is legal at that moment, which historical cases become training data, and which metric must beat the baseline. If the team reads the page and imagines different decisions, the next task is to repair the problem contract—not to choose an estimator.

Code

Write the learning contract before importing sklearn·python

contract = {
    "target": "will_churn_within_30_days",
    "prediction_time": "end of each calendar day",
    "legal_features": [
        "plan_tier",
        "tenure_days",
        "tickets_last_30d",
        "logins_last_7d",
    ],
    "metric": "PR-AUC, with recall >= 0.70 on high-risk users",
    "baseline": "current rules-based churn flag",
    "action_if_positive": "send retention offer next morning",
}
assert contract["baseline"], "every ML project needs a baseline to beat"

A minimal first model that respects the contract·python

from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
from sklearn.metrics import average_precision_score

X = customers[contract["legal_features"]]
y = customers[contract["target"]]

X_tr, X_te, y_tr, y_te = train_test_split(
    X, y, test_size=0.2, stratify=y, random_state=7
)
model = LogisticRegression(max_iter=200).fit(X_tr, y_tr)
pr_auc = average_precision_score(y_te, model.predict_proba(X_te)[:, 1])
print(f"baseline beat? PR-AUC={pr_auc:.3f}")

The four-piece contract

Learning is not memorization

The model is one system component

A good score can still lie

Write this before the first model

Code

External links

Exercise

Progress

Comments 0