JSONL Is Ground Truth

"The log is the truth. The database is a story the log tells — and can tell again."

The Log Is Canonical

Forge stores its data twice, and only one of them is the truth. Append-only JSONL op-logs under ~/cwk-dbs/forge-db/ record every mutation: a crumb added, a crumb revised, an interpretation written, an analysis artifact produced, a module changed. That log is ground truth. SQLite, running WAL, is a derived mirror that serves queries, aggregates, and the client. When you internalize which is which, a whole class of bugs disappears — because the database is never something you protect, only something you can rebuild.

Write Before Show

The ordering rule is strict: every event is durable in the JSONL before it is shown to Dad. Nothing appears on screen that has not first been written to the canonical log. This is the same 'write before show' discipline cwkPippa uses for conversations, and it is why a crash or a stopped process can never leave the truth behind the UI. The screen is downstream of the log, always — the log is not a record of what the screen showed; the screen is a view of what the log holds.

JSONL op-logs are ground truth, written before anything is shown; SQLite is a derived mirror. The database exists to be queried, not to be trusted as the source. Every crumb, revision, interpretation, artifact, and module change is an appended event first, and a folded row second.

Why Two Stores At All

If the log is the truth, why keep SQLite? Because a log is perfect for durability and terrible for questions. "Show me symptom sites this month" is a query, and queries want indexes and folded rows, not a replay of every event since the beginning. So Forge folds the log into SQLite for reading while keeping the log for truth — each store doing the job it is good at. The discipline is only to never confuse them: the mirror is fast and disposable; the log is slow to query and impossible to lose.

Code

The op-log is canonical; SQLite is folded from it·text

~/cwk-dbs/forge-db/  (append-only JSONL, ground truth)
  crumb.add       {id:01J8A, text:"knee stiff", target_date:...}
  interpretation.add {crumb:01J8A, types:[symptom], severity:mild}
  crumb.revise    {id:01J8A, target_date:<corrected>, revision:2}
  module.add      {id:mod_bike, kind:workout, name:"Indoor Bike"}
  artifact.add    {kind:window, source_crumb_ids:[...], body:...}
        |
        |  fold (write-before-show, then project)
        v
SQLite (WAL, derived mirror): crumbs, interpretations,
        modules, artifacts - indexed rows for fast queries

# The log is the truth. SQLite is a fast, disposable view of it.

Exercise

Suppose the office disk hiccups and a handful of SQLite rows end up corrupt. In Forge's model, how much health data did Dad just lose, and what exactly do you do to make the database correct again? Contrast that with an app where SQLite is the only store.

Hint

In Forge, zero data is lost — the JSONL still holds every event — and you rebuild the mirror from the log. In a SQLite-only app, the corrupt rows are the truth, and there is nothing to rebuild from.

The Log Is Canonical

Write Before Show

Why Two Stores At All

Code

External links

Exercise

Progress

Comments 0