Skip to content
C.W.K.
Lesson 03 of 05 · published

Rebuild on Every Start, No Migration

~14 min · sqlite, replay, projection, schema

Level 0Mise en Place
0 XP0/43 lessons0/12 achievements
0/100 XP to next level100 XP to go0% complete

Delete first, then replay

The first thing Sizzle's engine does when it starts is throw its database away. The FastAPI lifespan calls sizzle.open in a worker thread, which is one line: store.rebuild(journal.events()). The log then says how many journal events the mirror was rebuilt from.

rebuild closes any open connection and unlinks mirror.sqlite3 together with its -wal and -shm siblings, so no leftover state from the last run can leak in. It opens a fresh connection, runs the whole SCHEMA string with executescript, applies every event in journal order, and commits once at the end. Read the schema and you will notice there is no IF NOT EXISTS anywhere. It does not need one. The file it runs against is always empty.

Why there is no migration

Because the database is rebuilt from scratch on every start, the schema is just a Python string in backend/store.py. Changing it means editing that string and restarting. There is no version table, no ALTER TABLE, and no upgrade script that has to run against every old copy of the file. The house sign says it directly: the schema changes freely and no migration exists.

The hard part of a real migration is backfill: filling a new column for rows written before the column existed. Replay does that for free, as long as the journal kept the fact. Sizzle's journal often keeps more than the mirror reads. reading.recorded carries the brain, the effort and the resolved model, while the readings table stores only the brain. reading.requested carries a reason when a rubric bump asked for it, and no column holds that either. Add a model column tomorrow and the next start fills it for every reading ever recorded.

The price is start time. Every start reads and applies the whole journal. For a record that grows by human acts, that stays small. The rule is still worth knowing: replay cost grows with history, and an engine like this would need a snapshot the day that cost starts to show.

Principle: When the database is a projection, its schema is code. Change it like code and let replay do the backfill.

The event catalogue, counted from code

The house sign gives the grep: rg 'def _on_' backend/store.py. At this snapshot it finds 17 handlers, and each of the 17 types has a writer in the engine. Here is what each one does to the mirror:

EventWhat it does to the mirror
encounter.capturedInserts the plate, then each of its photos in order
encounter.revisedUpdates the plate's kind and note, whichever is present
reading.requestedSets the plate back to pending and clears the error
reading.recordedInserts the reading, marks the plate done, and makes this reading the current one
reading.failedMarks the plate failed with the error text
class.proposedInserts a class and its parent edges
class.parentedAdds one parent edge, ignored if it exists
class.revisedUpdates only the fields the event carries
class.adoptedSets the status to adopted
class.renamedSets the new name and the status to adopted
class.linkedInserts or replaces the link for that plate, class and source
correction.addedInserts Dad's correction
thread.boundBinds a Sizzle thread key to a cwkPippa conversation, once
turn.askedInserts a pending turn
turn.answeredStores the answer, emotion and actions, and marks the turn done
turn.failedMarks the turn failed with the error text
action.appliedRecords which of a turn's actions Dad applied, and does nothing if the turn is unknown

Four tables hold plates, photos, readings and corrections, three hold the class tree and its links, and two hold conversations. Tracks 3, 4 and 6 explain who writes which. What matters here is that this table is the whole mirror. Nothing reaches SQLite any other way.

Dispatch by name, refuse the unknown

_apply turns the event type into a method name, class.linked into _on_class_linked, and calls it. If no such method exists, it raises ValueError. That is the same stance as the torn middle line in lesson 2: an event the code cannot understand stops the start, instead of being skipped. It has one practical consequence worth remembering. Once a new event type has been written, an older checkout of the engine refuses to start on that journal. Rolling forward is free. Rolling back the code needs care.

Two entry points share _apply. The live path, apply, commits after every event, because a route is about to read what it just wrote. rebuild applies the whole journal in one transaction and commits once, so a long journal never pays for a commit per event. Where the same fact can legitimately arrive twice, the handler is written to absorb it: parent edges and thread bindings use INSERT OR IGNORE, and links use INSERT OR REPLACE.

What runs right after

The lifespan's order is deliberate. Rebuild the mirror. Declare the brain slots. Wire worker.kick into the domain. Start the reading worker, which first asks the mirror for every plate whose current reading predates RUBRIC_VERSION and records a reading.requested for each. Then resume conversation turns. A rubric bump re-reads every plate through the same journal. It edits nothing. The architecture doc keeps the one rule that makes all of this safe: never patch the mirror by hand.

If you have only met SQLite as a database you must migrate, sqlite-quest's Production SQLite track (its Production Migrations lesson) is the other side of this trade. Both are right. The question is only which file is the record.

Code

cwkSizzle, backend/store.py: Store.rebuild, delete the file, run the schema, replay, commit once (verbatim, lines 157-172)·python
    def rebuild(self, events: Iterable[dict[str, Any]]) -> int:
        with self._lock:
            if self._conn is not None:
                self._conn.close()
            for suffix in ("", "-wal", "-shm"):
                Path(str(self.path) + suffix).unlink(missing_ok=True)
            self.path.parent.mkdir(parents=True, exist_ok=True)
            self._conn = sqlite3.connect(self.path, check_same_thread=False)
            self._conn.row_factory = sqlite3.Row
            self._conn.executescript(SCHEMA)
            count = 0
            for event in events:
                self._apply(event)
                count += 1
            self._conn.commit()
            return count
cwkSizzle, backend/store.py: apply commits each live event; _apply dispatches by type and refuses an unknown one (verbatim, lines 188-197)·python
    def apply(self, event: dict[str, Any]) -> None:
        with self._lock:
            self._apply(event)
            self.conn.commit()

    def _apply(self, event: dict[str, Any]) -> None:
        handler = getattr(self, "_on_" + event["type"].replace(".", "_"), None)
        if handler is None:
            raise ValueError(f"unknown event type {event['type']}")
        handler(event["data"], event["at"])

External links

Exercise

In an app with an event journal and a rebuilt read model, add a column to one table for a fact your events already carry but no table stores yet. Make the change by editing only the schema and the one handler that fills it, then restart and check that old rows now have the value. Next, add an event type, write one, check out the previous commit, and watch what the old code does with the newer journal.
Hint
If your events never carried the fact, the backfill cannot happen, and that is the lesson too: decide what to record when you design the event. For the rollback, the failure you want is a loud refusal at startup. A quiet skip would hide the problem.

Progress

Progress is local-only — sign in to sync across devices.
Spotted a bug or have feedback on this page?Report an Issue

Comments 0

🔔 Reply notifications (sign in)
Sign in — Please sign in to comment.

No comments yet — be the first.