The Central Limit Theorem: Why Bells Show Up Everywhere

The Magic Trick

Take any distribution — uniform, exponential, weird custom thing. Sample from it many times. Compute the mean of each sample. The distribution of those sample means, regardless of the original distribution, will approach a normal distribution as your sample size grows.

Read that again. Regardless of the original distribution. That's the Central Limit Theorem. It's why the bell curve shows up in places that have nothing to do with bells.

Why Heights Are Bell-Shaped

Adult height is influenced by hundreds of genetic factors plus thousands of environmental ones. Each individually is a small random contribution. The CLT says: when you sum or average many independent random factors, the result tends toward a normal distribution. Heights are normal because they're the sum of many small factors. Same for measurement errors. Same for IQ. Same for sums of dice rolls.

Implications for ML

Why we assume Gaussian noise everywhere. Sensor noise, model residuals, batch averages — they're often sums of many tiny effects, so the CLT delivers a Gaussian-shaped result.
Why batch normalization works. Averaging activations over a batch invokes the CLT — the batch mean has a Gaussian-ish distribution.
Why confidence intervals on means are normal-based. Even if individual data is non-normal, the mean of enough samples is normal-ish.

The CLT is why "everything is roughly normal if you average enough." It's the universe's bias toward bell-shaped order in a chaotic raw world.

Code

Bell appears from non-bell·python

import numpy as np

# Take a wildly non-normal distribution — uniform [0, 1]
samples = np.random.uniform(0, 1, size=(100_000, 30))   # 100k samples of size 30

# Compute the mean of each sample
sample_means = samples.mean(axis=1)

# The distribution of sample means is approximately normal!
print(f"mean of means: {sample_means.mean():.3f}")  # ≈ 0.5
print(f"std of means : {sample_means.std():.3f}")   # ≈ 1/sqrt(12*30) ≈ 0.053

# Plot a histogram of sample_means — it's a bell, even though the
# original distribution was a flat uniform.

Exercise

Take 10,000 samples each of size 50 from an exponential distribution (very skewed, not bell-shaped). Compute each sample's mean. Plot a histogram of those means. The CLT predicts you'll see a bell.

Hint

np.random.exponential(scale=1.0, size=(10000, 50)).mean(axis=1). Despite the lopsided original distribution, the mean-of-means histogram is symmetric and bell-shaped.

The Central Limit Theorem: Why Bells Show Up Everywhere

The Magic Trick

Why Heights Are Bell-Shaped

Implications for ML

Code

External links

Exercise

Progress

Comments 0