Sequential Bayesian Updating: Belief That Evolves

"Belief that updates one piece of evidence at a time and ends up in the right place is the Bayesian engine of every honest reasoning system."

The Insight

Bayes' rule is most powerful when applied sequentially. Each new piece of evidence updates the prior into a posterior; that posterior then becomes the prior for the next piece of evidence. Over time, beliefs evolve coherently as data accumulates, regardless of where you started — provided you started with a prior that had at least some probability for the truth and provided your likelihoods are honest.

The Recursion

Day 1: prior(initial) + likelihood(evidence 1) → posterior(after evidence 1).
Day 2: prior(= posterior after 1) + likelihood(evidence 2) → posterior(after 1+2).
Day 3: prior(= posterior after 1+2) + likelihood(evidence 3) → posterior(after 1+2+3).

This is how diagnostic clinicians, intelligence analysts, weather forecasters, and good poker players actually reason. Not 'final verdict from one piece of evidence' but 'evolving belief as evidence accumulates.' The discipline is to update on each piece, not to wait for one decisive piece or to anchor on a first impression.

Convergence

An important consequence of sequential updating: with enough independent honest evidence, beliefs converge regardless of where they started. Two Bayesians with very different priors will converge to the same posterior as evidence accumulates, provided both update honestly. This is the Bayesian answer to 'how do we settle disagreements?' — accumulate enough evidence and the priors stop mattering.

The qualification 'enough independent honest evidence' is doing a lot of work. Convergence is slow when evidence is weak; the prior dominates for a long time. Convergence fails when evidence is correlated (the same evidence updated multiple times) or when one party is not updating honestly. Real-world disagreements often persist because one or both of these failure modes is operating.

The Operating Skill

The Bayesian discipline is not 'update once on the strongest evidence and decide.' It is 'update on each piece of evidence as it arrives, and let the cumulative posterior guide the next decision.' This is how diagnosis, forecasting, and skill acquisition actually work in practice — and it is the formal model of how honest reasoning under uncertainty should work in principle.

Code

Sequential updating: coin-bias learning·python

import numpy as np

# Sequential updating on a sequence of coin flips, trying to learn the bias.
# We start with a uniform prior over the bias parameter p (probability of heads).
# Each flip updates the belief.

p_grid = np.linspace(0.01, 0.99, 99)   # bias values to consider
prior = np.ones_like(p_grid) / len(p_grid)    # uniform prior

# True coin is biased toward heads (p_true = 0.7).
rng = np.random.default_rng(230)
p_true = 0.7

posteriors = [prior.copy()]
for n_flips in (5, 25, 100, 500):
    flips = rng.binomial(n=1, p=p_true, size=n_flips)
    heads = flips.sum()
    tails = n_flips - heads
    # Likelihood of the observed data given each candidate p.
    likelihood = p_grid ** heads * (1 - p_grid) ** tails
    posterior = prior * likelihood
    posterior /= posterior.sum()
    mean_estimate = (p_grid * posterior).sum()
    print(f"after {n_flips:>4} flips ({heads} H / {tails} T): "
          f"posterior mean estimate of p = {mean_estimate:.4f}")
    prior = posterior   # next round's prior is this round's posterior

print(f"\nTrue p = {p_true}")
print("Notice how the estimate converges toward the true value as more data arrives.")
# Even though we started with a uniform prior (saying 'we know nothing'),
# the posterior converges to the true p as evidence accumulates.
# A different starting prior would converge to the same answer eventually.

Exercise

Pick a belief you are currently uncertain about (a project's chance of success, a hypothesis you are testing, a person's character). Identify what evidence has accumulated so far. Then update sequentially: name your prior before any evidence, then update piece by piece. Notice whether your final posterior matches your gut. Often they differ — and the disciplined Bayesian posterior is usually more reliable than the gut.

Hint

Sequential updating forces you to assign weight to each piece of evidence individually, which prevents the gut-shortcut of overweighting the most recent or most vivid item.

The Insight

The Recursion

Convergence

The Operating Skill

Code

External links

Exercise

Progress

Comments 0