Z-Scores: Comparing Apples to Oranges

Different Units, a Common Scale

A z-score measures a value relative to a reference mean and standard deviation:

$z = \frac{x - μ}{σ}$

The original units cancel, and the sign records whether the value lies above or below the mean. A z-score of 1.5 means 1.5 standard deviations above the chosen reference mean. Comparisons remain meaningful only when the reference populations and measurements are themselves comparable.

When Does Standardization Produce a Standard Normal?

If $X$ is normally distributed, then $(X - μ) / σ$ follows the standard normal distribution with mean 0 and standard deviation 1. If $X$ is skewed, multimodal, or heavy-tailed, z-scoring preserves that shape. It standardizes location and scale; it does not manufacture normality.

Uses and Limits in ML

Standardization often helps optimization and distance-based models by putting features on comparable numerical scales. It is not necessary for every model, and an absolute threshold such as $∣ z ∣ > 3$ is only a sensible outlier rule when the reference distribution and application justify it.

Z-scores remove units and express distance in standard deviations. They do not erase distribution shape or make unrelated populations automatically comparable.

Code

Two scales become one·python

import numpy as np

scores_math = np.random.normal(70, 10, 100)
scores_physics = np.random.normal(60, 5, 100)

# Z-score each
z_math = (scores_math - scores_math.mean()) / scores_math.std()
z_physics = (scores_physics - scores_physics.mean()) / scores_physics.std()

print(f"math   z: mean={z_math.mean():.3f}, std={z_math.std():.3f}")
print(f"physics z: mean={z_physics.mean():.3f}, std={z_physics.std():.3f}")
# Both: mean ≈ 0, std ≈ 1 — directly comparable

Exercise

A student scores 85 in math (class μ=70, σ=10) and 65 in physics (class μ=55, σ=5). Compute z-scores. Which subject is the student doing better in, relative to peers?

Hint

z_math = (85-70)/10 = 1.5; z_physics = (65-55)/5 = 2.0. Physics is the better performance, even though the raw score is lower — z-scores reveal it.

Different Units, a Common Scale

When Does Standardization Produce a Standard Normal?

Uses and Limits in ML

Code

External links

Exercise

Progress

Comments 0