Independence Is a Tested Claim, Not a Default

"Independence is a property of a probability model, not a mood you attach to two variables because they look unrelated."

Two Equivalent Definitions

When P(B)>0, events A and B are independent if conditioning on B does not change A's probability:

P(A | B) = P(A)

Equivalently, P(A and B) = P(A) × P(B). The product definition also handles zero-probability edge cases without dividing by P(B).

Independent Is Not Mutually Exclusive

Mutually exclusive events cannot occur together, so their joint probability is zero. Two mutually exclusive events with positive probability therefore cannot be independent: observing one rules out the other. The caveat matters—if one event has probability zero, mutual exclusivity and independence can coexist mathematically.

Independence also does not mean "no causal relation" in ordinary language. It says that a specified joint distribution factorizes. Which variables, population, and conditioning information are included in the model all matter.

Do Not Assume It by Habit

Some real systems are well approximated as independent; others have serial dependence, shared causes, clustering, or sampling links. Neither "everything is connected" nor "everything is independent until proven otherwise" is a useful universal rule. State the assumption and test its consequences against the design and data.

A third variable can make two variables associated without one causing the other. Conversely, a weak sample correlation does not prove independence, because nonlinear or conditional dependence may remain.

The Rule That Saves You

Before multiplying probabilities, identify the joint model that licenses the product. Randomization, physical separation, a sampling design, or a defensible conditional model can provide that basis. A story alone cannot.

Where This Matters Later

The introductory Central Limit Theorem uses independent, identically distributed variables as a simple sufficient condition. More general versions allow some dependence, and correlated jointly normal variables still have normal sums. The real question is whether the dependence structure and variance calculation match the theorem and model being used.

Code

Independence: the textbook case versus the real-world case·python

import numpy as np
rng = np.random.default_rng(7)
N = 50_000

# Pair 1: coin flip and card draw — engineered to be independent.
coin = rng.integers(0, 2, size=N)               # 0 or 1
card_is_heart = rng.random(size=N) < 0.25       # 1/4 cards are hearts

p_coin = coin.mean()
p_heart = card_is_heart.mean()
p_both = (coin & card_is_heart).mean()
print(f"Pair 1 (independent by design):")
print(f"  P(coin) * P(heart) = {p_coin * p_heart:.4f}")
print(f"  P(coin AND heart)  = {p_both:.4f}   <- matches the product")

# Pair 2: synthetic height and weight — correlated through a shared cause.
# Both are driven by a hidden 'body size' variable.
body_size = rng.normal(size=N)
height_tall = (body_size + 0.3 * rng.normal(size=N)) > 0   # taller-than-median
weight_heavy = (body_size + 0.3 * rng.normal(size=N)) > 0  # heavier-than-median

p_tall = height_tall.mean()
p_heavy = weight_heavy.mean()
p_both2 = (height_tall & weight_heavy).mean()
print(f"\nPair 2 (correlated through hidden body size):")
print(f"  P(tall) * P(heavy) = {p_tall * p_heavy:.4f}")
print(f"  P(tall AND heavy)  = {p_both2:.4f}   <- much bigger than the product")

# The product rule for joint probability ONLY holds under independence.
# When it breaks, you have hidden coupling — and the world is full of it.

Exercise

Choose three pairs of variables from daily life. For each, define the population and time window, identify possible shared causes, and describe a design or diagnostic that would make an independence approximation more or less credible.

Hint

Check time trends, repeated measurements, clustering, selection, and conditioning variables. Independence is always relative to a stated model.

Two Equivalent Definitions

Independent Is Not Mutually Exclusive

Do Not Assume It by Habit

The Rule That Saves You

Where This Matters Later

Code

External links

Exercise

Progress

Comments 1