Classification Basics

~26 min · classification, logistic-regression

Level 0Scout

0 XP0/48 lessons0/11 achievements

0/120 XP to next level120 XP to go0% complete

Probabilities first, labels second

A classifier almost always produces a score; the label is a downstream decision (score above threshold). Treat the score (probability) as the model's true output. Threshold choice belongs in the design doc, not the training script.

Logistic regression as the baseline

Logistic regression is the linear classifier baseline. It is fast, calibrated by default for many problems, and the coefficients are interpretable as log-odds. It is rarely the best model on tabular data, but it sets the floor and helps you spot leakage early.

Multi-class, multi-label, ordinal

Multi-class picks one of K labels. Multi-label assigns any subset of K labels (separate sigmoid per label). Ordinal classification respects the ordering of labels ("bad < ok < great") and benefits from custom losses. Pick the framing before the model.

Code

Logistic regression with class weights and balanced output·python

from sklearn.linear_model import LogisticRegression
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import Pipeline

pipe = Pipeline([
    ("scale", StandardScaler()),
    ("clf", LogisticRegression(max_iter=1000, class_weight="balanced")),
])
pipe.fit(X_train, y_train)
probs = pipe.predict_proba(X_val)[:, 1]

Multi-label sigmoid head with OneVsRestClassifier·python

from sklearn.multiclass import OneVsRestClassifier
from sklearn.linear_model import LogisticRegression

tags_model = OneVsRestClassifier(LogisticRegression(max_iter=1000)).fit(X_train, Y_train_multilabel)

External links

Exercise

Train a logistic regression baseline on your classification problem. Report PR-AUC, recall at precision 0.7, and the operating threshold. Use this baseline as the bar every fancier model must beat by a meaningful margin.