The Central Limit Theorem: Why Bells Show Up Everywhere

What the Theorem Actually Approximates

In the classical central limit theorem, independent identically distributed variables with finite mean and variance have a standardized sum—or equivalently a properly centered and scaled sample mean—that approaches a normal distribution as sample size grows.

The conditions matter. Heavy-tailed variables can have infinite variance, dependence can change the limit, and a single dominant contribution can prevent the usual approximation. The theorem describes the sampling distribution of a sum or mean, not the shape of the original observations.

Why Normal Approximations Still Appear Often

Measurement error and aggregates can combine many modest effects, making a Gaussian approximation useful. But adult height, sensor noise, or residuals are not normal merely because the phrase “many factors” applies; biology, boundaries, correlation, and mixtures also shape the distribution. Check the approximation rather than invoking the theorem as a universal cause.

Implications for ML and Statistics

Sampling distributions: standard errors and confidence intervals for means often use a CLT when its conditions and sample size are adequate.
Noise models: Gaussian noise can be a defensible approximation for additive effects, but residual diagnostics still matter.
Batch normalization: it uses batch statistics to re-center and rescale activations. Its mechanism is not that the CLT makes activations Gaussian.

The CLT concerns standardized sums and means under stated conditions. It does not say that every distribution becomes normal when the dataset is large.

Code

Bell appears from non-bell·python

import numpy as np

# Take a wildly non-normal distribution — uniform [0, 1]
samples = np.random.uniform(0, 1, size=(100_000, 30))   # 100k samples of size 30

# Compute the mean of each sample
sample_means = samples.mean(axis=1)

# The distribution of sample means is approximately normal!
print(f"mean of means: {sample_means.mean():.3f}")  # ≈ 0.5
print(f"std of means : {sample_means.std():.3f}")   # ≈ 1/sqrt(12*30) ≈ 0.053

# Plot a histogram of sample_means — it's a bell, even though the
# original distribution was a flat uniform.

Exercise

Take 10,000 samples each of size 50 from an exponential distribution (very skewed, not bell-shaped). Compute each sample's mean. Plot a histogram of those means. The CLT predicts you'll see a bell.

Hint

np.random.exponential(scale=1.0, size=(10000, 50)).mean(axis=1). Despite the lopsided original distribution, the mean-of-means histogram is symmetric and bell-shaped.

What the Theorem Actually Approximates

Why Normal Approximations Still Appear Often

Implications for ML and Statistics

Code

External links

Exercise

Progress

Comments 0