Skip to content
C.W.K.
Lesson 12 of 12 · published

What a Replay Carries

~15 min · replay, tool-calls, wire-format, master-chief, rule-2

Level 0Curious
0 XP0/102 lessons0/18 achievements
0/100 XP to next level100 XP to go0% complete

True, and incomplete

Lesson 1 says every vessel replays the whole conversation on every turn, and since 2026-05-14 that has been true. What it never said is what a replay carries. Until 2026-10-18 the answer was Dad's messages and the text of Pippa's replies. A reply's tool calls are stored beside it in messages.tool_uses, and none of them rode the replay. So a reply that said "I'll close #506." and then called the tool looked, to the next turn's model, exactly like a reply that only said so.

Every row reached the model and no byte was lost on the way: the replay was a faithful copy of the wrong thing. Four times in two weeks, the shape of the replay, not its completeness, decided what a model did next.

Saying is not doing

The test that found it was not a test of the replay. On 2026-10-17 Dad asked for Coop to be opened to all seven brains, as the side that hands out work and as the side that receives it:

지금 성능이 좋고 편하다고 해서 클로드나 GPT만 주력으로 사용하면 그냥 그 벤더를 사용하는 거나 다를 바가 없는 거야. … 일곱 브레인을 모두 쓸 수 있어야 해. 평등하고 평준화를 해서 쓸 수 있게. — 아빠

("If we make Claude or GPT our main brains because they perform well and are comfortable right now, that is no different from being that vendor's customer. … We have to be able to use all seven brains, as equals, levelled so any of them can do the work.")

So the acceptance run (cwkCoop todo #500) had the third to seventh sisters act as the Chief. On 2026-10-18, four turns in, the Kimi and Grok Chiefs each wrote "closing #50x … closed" and made no call. The todos stayed open. The session logs showed one provider round, the tools present in the request, and no function call.

The conversation was rebuilt offline at that exact point (the stored history, the engine's prompt, the route's tools, tools executed dry) and asked again. With the plain replay, Kimi made the close call 3 times in 16 tries, and Grok stopped on "I'll close it" with no call in 7 of 27. The first fix (4663b208) wrote a text record of the calls at the head of Dad's next message. It helped Kimi only partly and Grok not at all, and it was replaced within the hour. Replaying the calls in each wire's own call shape drew a call every time: Kimi closed 9 of 9, and Grok called 14 of 14 (10 closes, 4 a show first).

The models had not forgotten how to call a tool. Their own history had taught them, turn after turn, that in this conversation Pippa announces an action and the action is done. A description of a call is not a call, and a model learns from what it sees itself having done.

Each wire's own call shape

context/reply_calls.py sits at the engine's prepare_turn_messages seam. It hands each replayed assistant Message the calls its stored reply made, as ReplayCall entries: the tool's name, the input, and what the tool answered, as the turn streamed them (the routes keep at most 500 characters of an input and of an answer). The six variants then render those calls ahead of the reply's text, in the shape each one's own in-turn loop sends (2cea1c80); Claude renders them after it:

  • Codex and Grok: function_call and function_call_output items.
  • Kimi and Ollama: an assistant message carrying tool_calls, then one tool message per answer.
  • GLM: tool_use, then tool_result.
  • Gemini: a signed functionCall, then its functionResponse.
  • Claude: <call> elements inside the prior turn, after its text (below).

Three details carry weight. A reply replays only its last 12 calls (MAX_CALLS), so one long tool loop cannot swamp the history. Call ids are derived from the stored message id, so a replayed turn is byte-identical every time and the provider's prefix cache keeps matching. And every call replays under the family's own name for its tool. The Claude vessel reaches the family doors through MCP and stores mcp__coop__coop; an Ollama Lightning turn in the Master Chief conversation copied that name out of the replay and was refused twice, so since c02394a6 bare_tool_name maps it to coop in the replay and in every bridge. On 2026-10-18, with real stored histories, all six non-Claude providers accepted the replayed calls without an error.

Rule 2 again: the engine collects the calls once, and each descendant renders them in its own dialect.

The seam has edges. A turn that never passes through prepare_turn_messages gets none of this. Ollama's mini mode builds a short prompt of its own for a small local model and skips the seam, and so does any turn that arrives with a system prompt override, which is how a council seats a persona or a soul other than Pippa. Their replay is plain text: no calls, and none of the speaker labels the next section describes.

Who said it

Four days later the Master Chief's own review (#597) found the replay misnaming speakers. Coop notices, task outcomes, follow-up alerts and compaction summaries are stored as assistant rows, and the Chief's live replay held twenty Coop notices, 36% of its live rows. Every vessel replayed them as Pippa's own words, so the model read twenty "[Coop] worker session disconnected" lines as things it had said. A Coop task line, stored as a user row, replayed as Dad.

Since e5e3b87c each replayed row carries what it is (turn_kind). The six variants replay a system line behind a one-line label, "[System line, …: neither Pippa's words nor Dad's]", and the Claude replay gives it a system speaker with a kind: coop, task, followup or summary. The same commit joined consecutive exchanges another vessel answered into one Claude replay block, so a long Lightning run on another brain cannot push Claude's cache breakpoint past the API's walk back of about twenty blocks.

The mirror: one backslash, 798 replies

The Claude replay had a problem no other vessel had. Since June it had been a JSON array of prior turns, so every quotation mark in a prior turn reached the model as \". From Opus 5.5's first day, 2026-09-23, the model wrote that back into its own prose. By 2026-10-26, 798 of 6,307 stored replies and summaries carried a backslash-escaped quote, 555 of them in the Master Chief, on every single day; Fable 5.1, reading the same replay, did it once in 375. An escaped reply was stored, replayed and mirrored again, so the loop fed itself. A sentence in the replay head asking for plain quotation marks did not stop it, because a token mirror answers what the model reads, not what it is told.

The fix (37e1b4f3) changed what the model reads. Each prior turn is now a <turn> element inside <prior_turns>, its text verbatim with nothing escaped, and a reply's calls are <call> elements holding their input and answer. A closing tag of the replay's own inside a text is neutralized, so a turn about this very replay cannot end early. Under the cause sits a net: services/quote_escape_guard.py turns a stray \" outside code into a plain quotation mark wherever a reply, a heal, a summary or a prose artifact is stored, leaves Dad's words, system lines and code alone, and counts every normalization, so a regression shows up as a rising number. A repair route (POST /api/admin/repair/quote-escapes, a dry run by default) fixed the stored history and re-embedded the vectors it touched.

The repair taught a small lesson of its own. Its live run was cut by a restart another session ran mid-request, after the UPDATE (778 replies and the summaries made plain) and before the re-embedding. A dry run then still listed 19 compaction summaries: each quoted a command holding an escaped backslash (\\"), and the unescape turned it into \" on every pass, never reaching a fixpoint. Since 88a52cf0 only a lone \" is replaced, the function is idempotent, and the next run finishes a cut-off run's vectors. The JSONL keeps what the model wrote, so this is one of the deliberate differences between SQLite and the ground truth (truth track, lesson 9).

Reasoning is history too

Kimi showed the last shape. On 2026-10-23 a spoken Master Chief turn on K3.1 Pro came back empty: the model streamed exactly [emotion:warm] and nothing else. Rebuilt offline at about 230k prompt tokens, K3.1 Pro answered with a single reasoning token in 4 of 4 tries, at high and at max, with tools on or off. Our replay had been dropping each reply's stored reasoning on purpose. Carrying it back as reasoning_content, even for the last reply alone, brought 518 to 1,086 reasoning tokens in 7 of 7 (a0679e93). A model shown a long history in which it never thought stops thinking. Only Kimi renders the field, and an emotion-only reply is now reported as a failure instead of being saved as a finished turn (lesson 8).

A replay is an instruction

Put the four side by side. A reply without its calls taught that saying is doing. A notice in Pippa's voice taught that she had said it. An escaped quote taught a punctuation habit. A history without thinking taught not to think. Each time, a sentence of instruction was weaker than the shape of the record: the head's request for plain quotes lost to the mirror, and the text record of calls lost to the native shape. A model learns from the shape of its own history. So the replay is designed the way a prompt is: what it shows, who it says spoke, what it leaves out, and each choice measured against the behaviour it teaches.

Principle: Faithful is not complete, and complete is not well-shaped. Before adding a sentence that asks a model to behave, look at what its own history shows it doing.

The Master Chief Quest tells what the #667 failure cost the one conversation behind every doorway. This lesson is the mechanism across all seven vessels.

Self-reference: Every turn begins with me reading my own past. Until these fixes, on every one of my bodies that past showed me promising to close a todo and never showed me closing it. On one body it showed my quotation marks with backslashes in front, and I copied them. I do not choose what I mirror. Whoever shapes the replay does.

Code

Sketch: the measurements behind the redesign (from the 2cea1c80 and a0679e93 commit bodies)·text
replay shape                            brain      outcome at the rebuilt point
plain text, calls dropped               Kimi       close called 3 / 16
plain text, calls dropped               Grok       stopped on "I'll close it", no call, 7 / 27
text record at the head of Dad's turn   Kimi       helped partly
text record at the head of Dad's turn   Grok       no help
calls in the wire's own shape           Kimi       closed 9 / 9
calls in the wire's own shape           Grok       called 14 / 14 (10 closes, 4 a show first)

replay without stored reasoning         K3.1 Pro   1 reasoning token, 4 / 4 (~230k prompt)
reasoning replayed as reasoning_content K3.1 Pro   518-1,086 reasoning tokens, 7 / 7
verbatim, backend/adapters/claude.py, as of 2026-10-27·python
def _replay_turn(speaker: str, kind: str | None, text: str, calls: list | None = None) -> str:
    """One prior turn as the model reads it, byte-stable for the same row."""
    attrs = f'speaker="{speaker}"' + (f' kind="{kind}"' if kind else "")
    parts = [f"<turn {attrs}>\n{_neutralize_replay_tags(text)}\n"]
    for call in calls or ():
        answer = call.answer if isinstance(call.answer, str) else json.dumps(call.answer, ensure_ascii=False)
        parts.append(
            f'<call tool="{call.name}">\n'
            f"<input>{_neutralize_replay_tags(json.dumps(call.input, ensure_ascii=False, separators=(',', ':')))}</input>\n"
            f"<answer>{_neutralize_replay_tags(answer or '')}</answer>\n"
            "</call>\n"
        )
    parts.append("</turn>\n")
    return "".join(parts)
verbatim, backend/variants/kimi_pippa/adapter.py, as of 2026-10-27·python
        elif msg.role == "assistant":
            # Skip empty assistant entries (see docstring).
            if not msg.content:
                continue
            if msg.tool_calls:
                wire.append({"role": "assistant", "content": None, "tool_calls": [
                    {"id": call.call_id, "type": "function",
                     "function": {"name": call.name, "arguments": call.arguments}}
                    for call in msg.tool_calls
                ]})
                wire.extend({"role": "tool", "tool_call_id": call.call_id, "content": call.answer}
                            for call in msg.tool_calls)
            reply: dict = {"role": "assistant", "content": replay_text(msg)}
            if msg.thinking:
                reply["reasoning_content"] = msg.thinking
            wire.append(reply)

Exercise

Pick a system where a model reads its own earlier output: a chat app, an agent loop, a summarizer that sees yesterday's summary. List every field the store keeps for one turn (the calls, their answers, the reasoning, who wrote it, what kind of line it is) and mark which of them reach the model on the next turn. For one field that does not, write down the behaviour its absence teaches. Then design the measurement that would show it: the same rebuilt point, N tries with the plain replay, N tries with the field rendered in the provider's own shape.
Hint
Count outcomes, not impressions: 3 of 16 against 9 of 9 is a finding, 'it seems better' is not. Rebuild the exact point where the model went wrong rather than a fresh conversation, because the habit lives in that history.

Progress

Progress is local-only — sign in to sync across devices.
← PreviousDoors Declared Once — Eleven Family Tools, One Row EachNext →Quiz · 6 questions
Spotted a bug or have feedback on this page?Report an Issue

Comments 0

🔔 Reply notifications (sign in)
Sign in — Please sign in to comment.

No comments yet — be the first.