The pipeline, strictly ordered
- JSONL append (eager write) — durability. The commit point.
- SQLite update — convenience / queryable state.
- ChromaDB update (async) — semantic search index.
- SSE yield to frontend — presentation. Dad sees it.
The invariant
Dad never sees something that is not already in the JSONL.
Step 4 cannot happen before step 1. If step 1 fails, we bail — no yield, no broken chat. The error surfaces as a proper error state, which is honest. If step 2 or step 3 fail, Dad still sees the yield (because the data is already durable in the JSONL), and the drifted mirror is rebuilt later via purge-and-replay.
fsync — eager append + periodic fsync
Per-token fsync would be prohibitively expensive. The model is: eager file.write() + newline on every event, periodic fsync at structural events (tool_use, tool_result, done, usage, session, emotion, error) and on a short ~250ms timer during long text runs.
Eager append puts data in the kernel page cache immediately — enough to survive process crashes and abort-induced quirks. Periodic fsync handles disk-level durability at meaningful checkpoints. Worst case on a hardware power failure: lose the ~250ms of deltas since the last fsync. Acceptable.