Joint: Two Coins Are Not One Coin Twice

"Marginal, joint, and conditional probabilities answer different questions. Name the event before choosing the arithmetic."

Three Different Objects

Marginal: P(A), without fixing another event.
Joint: P(A and B), the probability that both occur.
Conditional: P(A | B), the probability of A within cases where B occurs.

They are linked by P(A and B) = P(A | B) × P(B). This identity does not assume independence.

AND and OR

For an AND event, use the multiplication rule with a conditional probability. Only under independence does it simplify to P(A) × P(B). The chance of rain and the chance that you forget an umbrella are not enough by themselves to compute the chance of both, because weather may change your behavior.

For an OR event, use P(A or B) = P(A) + P(B) − P(A and B). Subtract the overlap because it was counted twice. If A and B are mutually exclusive, the overlap is zero and simple addition is valid.

The Chain Rule

For several events, factor a joint probability into conditionals:

P(A and B and C) = P(A | B and C) × P(B | C) × P(C)

Different valid orderings give the same joint probability when the conditionals are defined consistently. The chain rule makes dependence explicit instead of quietly replacing it with independence.

Joint Space

A joint distribution keeps every combination and its probability visible. Two fair independent coin flips have outcomes HH, HT, TH, and TT, each with probability 0.25. The marginals are obtained by summing the appropriate joint cells.

Where This Matters Later

A likelihood uses the joint probability mass or density of the observed data under a parameter or hypothesis, viewed as a function of that parameter. For independent observations it factorizes into a product; for dependent observations the conditional structure must be modeled. A likelihood is not generally a probability distribution over parameter values, and a density value is not itself the probability of one exact continuous observation.

Code

Joint, marginal, and the chain rule, side by side·python

import numpy as np
rng = np.random.default_rng(2)

# A 2-by-2 joint distribution: rain (R) vs umbrella forgotten (F).
# Suppose these two are NOT independent — you forget the umbrella more often
# when the morning is sunny (because you don't bother grabbing it).
N = 100_000
rain     = rng.random(N) < 0.30   # P(R)  = 0.30
# Forgetting is correlated with NOT raining (you only grab the umbrella when
# the sky looks threatening).
forgot   = np.where(rain, rng.random(N) < 0.10, rng.random(N) < 0.50)

P_R  = rain.mean()
P_F  = forgot.mean()
P_RF = (rain & forgot).mean()
P_F_given_R = forgot[rain].mean()

print(f"Marginal P(rain)              = {P_R:.3f}")
print(f"Marginal P(forgot umbrella)   = {P_F:.3f}")
print(f"Joint    P(rain AND forgot)   = {P_RF:.3f}")
print(f"Chain    P(forgot|rain)*P(rain) = {P_F_given_R * P_R:.3f}  <- matches joint")
print(f"Product  P(rain)*P(forgot)    = {P_R * P_F:.3f}            <- WRONG: assumes independence")

# The chain-rule version matches the empirical joint.
# The product-of-marginals version is off — that's the cost of assuming
# independence when there's actually coupling.

Exercise

For your city's morning weather, estimate P(rain), P(carry umbrella | rain), and P(rain and carry umbrella). Check that the joint is no larger than either marginal, and explain what behavior would make rain and umbrella-carrying dependent.

Hint

Joint is always 'at most as common as either ingredient on its own.' If joint > marginal, you've mixed up the formula direction.

Three Different Objects

AND and OR

The Chain Rule

Joint Space

Where This Matters Later

Code

External links

Exercise

Progress

Comments 0