Sequential Bayesian Updating: Belief That Evolves

"Belief that updates one piece of evidence at a time and ends up in the right place is the Bayesian engine of every honest reasoning system."

The Insight

Bayes' rule is most powerful when applied sequentially. Each new piece of evidence updates the prior into a posterior; that posterior then becomes the prior for the next piece of evidence. Sequential updating is algebraically coherent when the joint model is specified correctly. Convergence also requires conditions such as identifiability, adequate prior support, informative data, and a well-specified likelihood; it is not guaranteed by sincerity.

The Recursion

Day 1: prior(initial) + likelihood(evidence 1) → posterior(after evidence 1).
Day 2: prior(= posterior after 1) + likelihood(evidence 2) → posterior(after 1+2).
Day 3: prior(= posterior after 1+2) + likelihood(evidence 3) → posterior(after 1+2+3).

These fields often revise judgments as evidence accumulates, sometimes with formal Bayesian models and sometimes with other methods. Not 'final verdict from one piece of evidence' but 'evolving belief as evidence accumulates.' The discipline is to update on each piece, not to wait for one decisive piece or to anchor on a first impression.

Convergence

An important consequence of sequential updating: under regularity and identifiability conditions, sufficiently informative data can reduce the influence of many different priors. Priors need not become irrelevant in weakly identified, misspecified, high-dimensional, or finite-data problems.

The qualification 'enough independent honest evidence' is doing a lot of work. Convergence is slow when evidence is weak; the prior dominates for a long time. Dependence does not invalidate Bayes when it is modeled in the joint likelihood; treating correlated reports as independent double-counts evidence. Real-world disagreements often persist because one or both of these failure modes is operating.

The Operating Skill

The Bayesian discipline is not 'update once on the strongest evidence and decide.' It is 'update on each piece of evidence as it arrives, and let the cumulative posterior guide the next decision.' This is how diagnosis, forecasting, and skill acquisition actually work in practice — and it is the formal model of how honest reasoning under uncertainty should work in principle.

Code

Sequential updating: coin-bias learning·python

import numpy as np

# Sequential updating on a sequence of coin flips, trying to learn the bias.
# We start with a uniform prior over the bias parameter p (probability of heads).
# Each flip updates the belief.

p_grid = np.linspace(0.01, 0.99, 99)   # bias values to consider
prior = np.ones_like(p_grid) / len(p_grid)    # uniform prior

# True coin is biased toward heads (p_true = 0.7).
rng = np.random.default_rng(230)
p_true = 0.7

posteriors = [prior.copy()]
for n_flips in (5, 25, 100, 500):
    flips = rng.binomial(n=1, p=p_true, size=n_flips)
    heads = flips.sum()
    tails = n_flips - heads
    # Likelihood of the observed data given each candidate p.
    likelihood = p_grid ** heads * (1 - p_grid) ** tails
    posterior = prior * likelihood
    posterior /= posterior.sum()
    mean_estimate = (p_grid * posterior).sum()
    print(f"after {n_flips:>4} flips ({heads} H / {tails} T): "
          f"posterior mean estimate of p = {mean_estimate:.4f}")
    prior = posterior   # next round's prior is this round's posterior

print(f"\nTrue p = {p_true}")
print("Notice how the estimate converges toward the true value as more data arrives.")
# Even though we started with a uniform prior (saying 'we know nothing'),
# the posterior converges to the true p as evidence accumulates.
# A different starting prior would converge to the same answer eventually.

Exercise

Pick a belief you are currently uncertain about (a project's chance of success, a hypothesis you are testing, a person's character). Identify what evidence has accumulated so far. Then update sequentially: name your prior before any evidence, then update piece by piece. Notice whether your final posterior matches your gut. Often they differ — and the disciplined Bayesian posterior is usually more reliable than the gut.

Hint

Sequential updating forces you to assign weight to each piece of evidence individually, which prevents the gut-shortcut of overweighting the most recent or most vivid item.

The Insight

The Recursion

Convergence

The Operating Skill

Code

External links

Exercise

Progress

Comments 0