Normalize vs Standardize: Two Ways to Tame Numbers

Same Goal, Different Vibes

You've got numbers. They live on wildly different scales — pixel values 0 to 255, ages 18 to 90, prices in dollars vs prices in won. Feeding them raw into a neural network is asking for trouble: the bigger features dominate gradients just because they're bigger. So we squash. There are two main flavors:

Normalization (Min-Max) — squeeze the values into a fixed range, usually $[0, 1]$ . Formula: $x_{norm} = \frac{x - x _{m i n}}{x _{m a x} - x _{m i n}}$ . Pixel values divided by 255 is the canonical example.
Standardization (Z-score) — shift and scale so the data has mean 0 and standard deviation 1. Formula: $z = \frac{x - μ}{σ}$ . This forces the data into the shape of a standard normal distribution.

When to Use Which

Normalization preserves shape. If your data was uniform-looking from 0 to 255, after dividing by 255 it's still uniform-looking from 0 to 1 — just smaller. Good for image pixels, where every pixel is structurally the same kind of thing.

Standardization changes shape too — it pulls outliers in but doesn't bound the result. Z-scores can be -3, +5, whatever. Good for tabular features that already kinda look normal-ish (ages, heights, salaries).

Normalization compresses range. Standardization aligns shape. Different surgeries — pick the one that matches what your data needs.

Why Your Network Cares

Neural networks update weights via gradients. If feature A ranges 0–1 and feature B ranges 0–1,000,000, the gradient w.r.t. B will dominate B's weight, making the network basically blind to A. Normalize, and they speak the same language. Skip the step, and you'll waste epochs (or the network just won't converge).

Code

Both flavors in 4 lines·python

import numpy as np

img = np.random.randint(0, 256, size=(64, 64, 3)).astype(np.float32)

# Min-max normalization → [0, 1]
img_norm = img / 255.0
print(img_norm.min(), img_norm.max())   # 0.0  1.0 (or close)

# Z-score standardization → mean 0, std 1
mean, std = img.mean(), img.std()
img_std = (img - mean) / std
print(img_std.mean().round(3), img_std.std().round(3))  # ~0.0  ~1.0

PyTorch flavor·python

import torch

# Same operations as PyTorch tensors — same vibe, GPU-ready
x = torch.randn(1000) * 50 + 100   # mean 100, std 50, very un-normalized

x_norm = (x - x.min()) / (x.max() - x.min())  # [0, 1]
x_std  = (x - x.mean()) / x.std()             # mean 0, std 1

Exercise

Take a 1-D NumPy array of 100 random integers in [0, 1000]. Apply (a) min-max normalization, (b) z-score standardization. Print mean & std for each. Which one has a guaranteed range? Which one has a guaranteed mean?

Hint

np.random.randint(0, 1001, size=100). Then divide and (subtract-then-divide). The 'guaranteed' answers are 'normalization' and 'standardization' respectively — but stating it isn't the lesson; *seeing* it on numbers is.