Regression Metrics: MAE, RMSE, R²

~26 min · metrics, regression

Level 0Scout

0 XP0/48 lessons0/11 achievements

0/120 XP to next level120 XP to go0% complete

Pick the metric that matches the business cost

MAE (mean absolute error) is the average error magnitude. RMSE (root mean squared error) punishes large errors quadratically. R² is the fraction of variance explained, comparable across datasets but unitless. Pick MAE when small and large errors hurt proportionally, RMSE when large errors hurt much more, and R² when you need a comparison metric across very different problems.

Symmetric vs asymmetric costs

Many real problems have asymmetric error costs. Predicting too few units of inventory costs lost sales; too many costs storage. Use quantile regression or a custom loss that mirrors the asymmetry instead of pretending MAE captures it.

The MAPE warning

Mean absolute percentage error (MAPE) is intuitive but breaks badly when the target is near zero or has zeros. Use SMAPE, weighted MAPE, or absolute error in the original units when stakeholders insist on a percentage.

Code

All three metrics in one block·python

from sklearn.metrics import mean_absolute_error, mean_squared_error, r2_score
import numpy as np

preds = model.predict(X_test)
mae = mean_absolute_error(y_test, preds)
rmse = np.sqrt(mean_squared_error(y_test, preds))
r2 = r2_score(y_test, preds)
print(f"MAE={mae:.3f}  RMSE={rmse:.3f}  R²={r2:.3f}")

Quantile regression for asymmetric costs·python

from sklearn.linear_model import QuantileRegressor

# Predict the 90th percentile so we under-stock less often
model = QuantileRegressor(quantile=0.9, alpha=0.1).fit(X_train, y_train)

External links

Exercise

For your regression problem, write down what an error of size 1 costs and what an error of size 10 costs. Pick the metric (MAE, RMSE, quantile, custom) that mirrors that cost shape. Justify why R² is or is not enough on its own.

Progress

Progress is local-only — sign in to sync across devices.

← PreviousLinear Regression Intuition Next →Underfitting, Overfitting, and Regularization

Spotted a bug or have feedback on this page?Report an Issue

Comments 0

🔔 Reply notifications (sign in)

No comments yet — be the first.