cwkPippa learned this in production: every event (prompt, delta, tool use, tool result, final usage) lands in JSONL keyed by conversation_id, encrypted at rest, append-only. SQLite and ChromaDB are derived mirrors — fast indexes for the application, rebuildable from the JSONL. Recovery from any inconsistency is purge-and-replay, not patch-and-reconcile.
Write before show
The reliability invariant: persist each delta to JSONL before rendering it to the user or surfacing it via API. A crashed stream still has every visible token on disk. Pair line-buffered writes with explicit flush so a SIGTERM mid-write does not lose the last line.
Healing on read
cwkPippa runs _heal_incomplete_turns on every conversation GET — rebuilds aborted turns from delta events, normalizes timestamps so order is preserved. The frontend never sees a broken turn. Healing is the cost of letting JSONL be canonical: derived state can drift, and the read path must repair it.
Principle: Single source of truth, durable, append-only. Indexes and caches are derived. Recovery is rebuild, not patch.
Code
Write before show·python
import json, pathlib
async def stream_with_persistence(conv_id: str, stream):
p = pathlib.Path(f"/srv/sessions/{conv_id}.jsonl").open("a", buffering=1)
try:
async for event in stream:
# Persist FIRST.
p.write(json.dumps({"event": event.dict()}) + "\n")
p.flush()
# Then render to the user.
yield event
finally:
p.close()
Rebuild SQLite from JSONL on inconsistency·python
import json
def rebuild_session(conn, conv_id: str, jsonl_path: str):
# Purge derived state for this session.
conn.execute("DELETE FROM messages WHERE conversation_id = ?", (conv_id,))
# Replay from JSONL.
for line in open(jsonl_path):
ev = json.loads(line)
if ev["event"]["type"] == "message_complete":
conn.execute(
"INSERT INTO messages (conversation_id, role, content, created_at) VALUES (?, ?, ?, ?)",
(conv_id, ev["event"]["role"], ev["event"]["text"], ev["event"]["created_at"]),
)
conn.commit()
For one chat-style feature, route every event to JSONL before rendering. Add a tool that rebuilds your DB rows for a session from the JSONL. Run rebuild against a known session and confirm the result matches the live DB.
Hint
If the rebuild differs from live, JSONL has bugs (or a write was missed). Fix the writer first; the rebuild is the integrity check.
Progress
Progress is local-only — sign in to sync across devices.