The Healing Layer — Reconstruction + Normalization · Pippa Quest

What healing does

On every conversation read, three repair passes run, in this order: _cleanup_empty_assistant_turns first, because an empty assistant shell would otherwise block the heal; then _heal_incomplete_turns; then _cleanup_dangling_parent_ids. Between the last two, any assistant row missing its tool-use record is backfilled from the JSONL.

Reconstruction

Walk the JSONL, find any user message with no completed assistant child, accumulate the streaming deltas (keyed by pippa.user_msg_id), write a synthesized assistant row to SQLite. Timestamp it at user.created_at + 1µs, not at heal-time — so ORDER BY created_at produces the natural chain.

Normalization

Cleanup any parent_id that doesn't reference an existing message, re-parent it to the previous message in created_at order. This kills the ghost-branch mechanism at its root — removing the dangling references entirely so no future placeholder can match against them.

The principle: The persistence layer is the long-lived part of the system. Bugs in client code can leave permanent footprints in it. Healing is the last line of defense against those footprints reaching presentation.

Why on every read

Healing runs idempotently on every GET — except while a stream is live on that conversation, because healing a turn that is still being written would race its writer. There's no separate 'repair' button anymore. The cost is low (it's reading the JSONL anyway), and it means corrupted state from any past version of any client can never reach the screen.

Code

The healing pass — sketch·python

async def get_conversation(conversation_id: str) -> Conversation:
    async with _heal_lock_for(conversation_id):
        messages = await store.get_messages(conversation_id)

        # 1. An empty assistant shell would block the heal below — clear it first.
        if await _cleanup_empty_assistant_turns(conversation_id, messages):
            messages = await store.get_messages(conversation_id)

        # 2. Rebuild any unfinished assistant turn from the JSONL.
        #    (A no-op while a stream is live on this conversation.)
        if await _heal_incomplete_turns(conversation_id, messages):
            messages = await store.get_messages(conversation_id)

        # 3. Re-point any parent_id that no longer resolves.
        if await _cleanup_dangling_parent_ids(conversation_id):
            messages = await store.get_messages(conversation_id)

    # Each pass returns True when it changed something; the caller re-reads
    # instead of trusting a pass to hand back rows it edited in SQLite.
    return Conversation(id=conversation_id, messages=messages)

The Healing Layer — Reconstruction + Normalization

What healing does

Reconstruction

Normalization

Why on every read

Code

Progress

Comments 0