C.W.K.
Lesson 03 of 06 · published

Write-Before-Show — Eager Append, Periodic fsync

~12 min · fsync, durability

Level 0Curious
0 XP0/52 lessons0/16 achievements
0/100 XP to next level100 XP to go0% complete

The pipeline, strictly ordered

  1. JSONL append (eager write) — durability. The commit point.
  2. SQLite update — convenience / queryable state.
  3. ChromaDB update (async) — semantic search index.
  4. SSE yield to frontend — presentation. Dad sees it.

The invariant

Dad never sees something that is not already in the JSONL.

Step 4 cannot happen before step 1. If step 1 fails, we bail — no yield, no broken chat. The error surfaces as a proper error state, which is honest. If step 2 or step 3 fail, Dad still sees the yield (because the data is already durable in the JSONL), and the drifted mirror is rebuilt later via purge-and-replay.

fsync — eager append + periodic fsync

Per-token fsync would be prohibitively expensive. The model is: eager file.write() + newline on every event, periodic fsync at structural events (tool_use, tool_result, done, usage, session, emotion, error) and on a short ~250ms timer during long text runs.

Eager append puts data in the kernel page cache immediately — enough to survive process crashes and abort-induced quirks. Periodic fsync handles disk-level durability at meaningful checkpoints. Worst case on a hardware power failure: lose the ~250ms of deltas since the last fsync. Acceptable.

Progress

Progress is local-only — sign in to sync across devices.