Purge and Replay

"When the mirror is wrong, you do not fix the mirror. You drop it and let the log draw it again."

The One Recovery Move

Because SQLite is derived, recovery has exactly one shape: purge and replay. If the mirror is ever wrong or corrupt, Forge drops the affected rows and rebuilds them from the JSONL. There is no diff-and-patch, no incremental reconciliation logic anywhere in the system. This is inherited straight from cwkPippa's hardest-won rule: do not patch derived state. Patching means enumerating every way the two stores could have diverged, and you will always miss one — so the missed case becomes a bug that corrupts the mirror more quietly than before.

A Schema Change Is the Same Move

The elegant part is that a schema change and a corruption recovery are the same operation. When the mirror's shape needs to change, Forge bumps db.SCHEMA_VERSION, and that bump is the migration: drop the mirror, replay the log into the new shape. You saw this twice already without naming it. Adding meal_slot was a SCHEMA_VERSION=3 drop-and-replay; adding the module kind field was SCHEMA_VERSION=4, and pre-kind events simply replayed as workout. No hand-written ALTER, no backfill script to get subtly wrong — the log already holds the truth, so the new mirror is just a fresh projection of it.

Recovery is purge-and-replay, never patch-and-reconcile. The SCHEMA_VERSION bump IS the migration: drop the derived mirror, replay the JSONL into the new shape. There is no incremental repair logic anywhere, because incremental repair is exactly where the subtle, quiet corruption lives.

Why This Is Safe by Construction

Replay is deterministic and idempotent: the same log folded into the same schema yields the same mirror, every time, no matter how many times you run it. That is what makes dropping the database a calm operation rather than a scary one — you are not deleting data, you are discarding a cache you can always regenerate. The original words, dates, and readings live untouched in the append-only log. The mirror is allowed to be wrong precisely because it is never the thing you are protecting.

Code

Purge-and-replay, and why patch-and-reconcile is banned·python

# BANNED: diff the two stores and patch the differences
def reconcile(sqlite, jsonl):
    for row in sqlite:
        if diverges(row, jsonl):
            patch(row)          # must handle EVERY divergence case
    # miss one case -> a quieter corruption than you started with

# THE ONLY MOVE: drop the mirror, replay the log
def rebuild(jsonl, schema_version):
    drop_all_derived_tables()
    create_tables(schema_version)
    for event in read_jsonl(jsonl):    # deterministic, idempotent
        fold(event)                    # same log -> same mirror

# A schema change is not special - it is rebuild() with a new
# schema_version. SCHEMA_VERSION bump IS the migration.

Exercise

A schema change needs to add a new derived field to the mirror. Describe how Forge migrates, and contrast it with the usual ALTER TABLE plus a backfill script. Name one class of bug the drop-and-replay approach makes structurally impossible.

Hint

Forge bumps SCHEMA_VERSION and rebuilds the mirror from the log — no ALTER, no backfill. The bug it removes: a backfill that computes the new field slightly wrong and leaves the mirror permanently inconsistent, with no clean source to notice against.

The One Recovery Move

A Schema Change Is the Same Move

Why This Is Safe by Construction

Code

External links

Exercise

Progress

Comments 0