What Regression Means

~24 min · regression, framing

Level 0Scout

0 XP0/48 lessons0/11 achievements

0/120 XP to next level120 XP to go0% complete

Continuous targets, real consequences

Regression means predicting a number on a continuous (or count) scale: price, demand, latency, revenue, days-until-churn. The metric is some form of distance between the prediction and the truth. Unlike classification, the magnitude of the error matters as much as the direction.

Two questions to ask first

What does an error of size E cost? Predicting 100 when the truth is 110 might be free in one business and disastrous in another.
Is the target naturally bounded? Counts are non-negative integers. Probabilities live in [0, 1]. Ignore the bound and the model will produce nonsense outside it.

The log trick

When the target spans orders of magnitude (revenue, web traffic, latency), predict log(target). Errors then correspond to ratios, which is usually what the business cares about. Remember to invert before reporting.

Code

Predict log-target when the scale is multiplicative·python

import numpy as np
from sklearn.linear_model import Ridge
from sklearn.compose import TransformedTargetRegressor

model = TransformedTargetRegressor(
    regressor=Ridge(alpha=1.0),
    func=np.log1p,
    inverse_func=np.expm1,
).fit(X_train, y_train)

Bounded targets need bounded models·python

# For non-negative targets, use Poisson or Tweedie regression
from sklearn.linear_model import PoissonRegressor

model = PoissonRegressor(alpha=1.0).fit(X_train, y_train_counts)

External links

Exercise

For one regression problem, sketch the cost of an error of size 1 and an error of size 10. Decide whether the cost is linear, quadratic, or asymmetric. Pick the matching loss (MAE, MSE, quantile, custom).