The Replay Remembers What She Did

Saying Is Not Doing

On 2026-10-18 the family ran a live acceptance. Each of the five brains other than Claude and GPT took a turn as chief of a fresh Pippa conversation and had to see Coop work through to the end with nothing but the coop tool (track 5, lesson 6). Four turns in, the Kimi chief and the Grok chief each wrote something like "closing #506 ... closed". Neither had called anything, and the todos stayed open.

The models were doing what the replay had taught them. Every vessel replayed prior turns as text: Dad's messages and Pippa's reply texts. The tool calls a reply made were stored right beside it, in messages.tool_uses, and never rode the replay. So in the history the model read, an earlier reply that said "I'll close #505." and then reported the result looked exactly like a reply that only said so. A few turns of that and the lesson is learned: in this conversation, closing is something you say.

Measured Before It Was Fixed

The fix was chosen by measurement. The session rebuilt that exact point offline: the stored history, the engine's prompt and the route's tools, with the tools executed dry. Then it asked for the next turn many times under three kinds of replay.

Replay	Kimi	Grok
Text only, as it was	3 close calls in 16 tries	stopped on "I'll close it" with no call in 7 of 27
A text record of the calls at the head of Dad's next message (the first fix)	helped only partly	did not help
The calls replayed in each wire's own call shape	closed 9 of 9	called 14 of 14 (10 closes, 4 a `show` first)

The middle row teaches the most. The first fix, shipped the same morning, described each earlier call in prose ahead of Dad's next turn. The facts were right and the form was wrong. A model trained on tool use learns what calling looks like from the call structure itself, and a paragraph that reports a call is still a paragraph. The text record was replaced the same day.

Each Vessel, Its Own Shape

context/reply_calls.py sits at the seam every vessel already shares, prepare_turn_messages. For each replayed assistant message it reads that reply's stored calls, pairs each call with the answer stored after it, and hands them to the vessel as Message.tool_calls. Each vessel renders them ahead of the reply's text the way its own in-turn loop sends them:

Codex and Grok: function_call and function_call_output items.
Kimi and Ollama: an assistant message carrying tool_calls, then tool messages.
GLM: tool_use, then tool_result.
Gemini: a model turn of functionCall parts, the first carrying a synthetic thought signature because Gemini rejects a call without one, then a turn of functionResponse parts.
Claude: a calls field in its prior-turn JSON, explained in the replay head.

A provider check that day sent real stored histories with replayed calls to Grok, Codex, Kimi, GLM, Gemini and Ollama, and all six took them without an error.

The Details That Keep It Safe

Byte-stable ids. A replayed call needs an id, and a fresh random one would change the history's bytes every turn and throw away the prompt cache from lesson 2. The ids are built from the stored message id (replay_<stem>_<n>), so a replayed turn is the same bytes every time it is replayed.

Bounded. A reply replays its last 12 calls. Inputs and answers are what the turn streamed, and the routes keep at most 500 characters of each; an input cut short of valid JSON travels as {"input_excerpt": ...} instead of as broken JSON. Only a reply with text is replayed at all, as before.

One vocabulary. The Claude vessel reaches the family tools through MCP and stores a call as mcp__coop__coop. An Ollama turn in Master Chief copied that name straight from the replay, called a tool it did not have under that name, and was refused twice. Calls now replay under the family's own name for the tool, coop, whichever vessel made them.

The replay teaches whatever it shows. A model reads its own history as examples of how this conversation works. If the record keeps what was said and drops what was done, the next turn learns to say. Replay actions in the form actions take, not as a description of them.

Code

Verbatim excerpt from context/reply_calls.py: one stored reply's calls, paired with their answers, ids derived from the message id·python

def reply_calls(tool_uses: Any, message_id: str) -> list[ReplayCall]:
    """The calls one stored reply made, each paired with its answer.

    A stored ``result`` entry is the answer to the call before it.
    """
    pairs: list[list[Any]] = []
    for use in normalize_tool_uses(tool_uses):
        if use["tool_name"] == "result":
            if pairs and pairs[-1][2] is None:
                pairs[-1][2] = use["tool_input"]
            continue
        # One vocabulary across vessels: a Claude turn's ``mcp__coop__coop``
        # replays as ``coop``, the name every other vessel calls (todo #530).
        pairs.append([bare_tool_name(use["tool_name"]), use["tool_input"], None])
    stem = _ID_UNSAFE.sub("", str(message_id))[:12] or "reply"
    calls = []
    for index, (name, raw, answer) in enumerate(pairs[-MAX_CALLS:]):
        tool_input = _as_object(raw)
        calls.append(ReplayCall(
            call_id=f"replay_{stem}_{index}",
            name=name,
            input=tool_input,
            arguments=json.dumps(tool_input, ensure_ascii=False),
            answer=answer or "",
        ))
    return calls

A minimal sketch you can run: the same two replies replayed as words only, then with their calls (stdlib, Python 3.9+)·python

"""A minimal sketch: replay a reply's tool calls, not only its words."""
import json

# Two stored replies. The first called a tool; the second only talked about it.
ROWS = [
    {"id": "m1", "role": "user", "text": "Close todo #12, the build is green."},
    {"id": "m2", "role": "assistant", "text": "Closed #12.",
     "tool_uses": [{"name": "coop", "input": {"action": "close", "id": 12}, "answer": "status: closed"}]},
    {"id": "m3", "role": "user", "text": "And #13?"},
    {"id": "m4", "role": "assistant", "text": "I'll close #13 now.", "tool_uses": []},
]
MAX_CALLS = 12


def plain_replay(rows):
    """What every vessel sent before 2026-10-18: the words alone."""
    return [{"role": r["role"], "content": r["text"]} for r in rows]


def replay_with_calls(rows):
    """The same history in a Responses-style wire: each stored call and its
    output ride ahead of the reply's text. Call ids come from the message id,
    so the bytes are identical on every turn and a prompt cache keeps matching."""
    items = []
    for r in rows:
        for i, use in enumerate(r.get("tool_uses", [])[-MAX_CALLS:]):
            call_id = f"replay_{r['id']}_{i}"
            items.append({"type": "function_call", "call_id": call_id, "name": use["name"],
                          "arguments": json.dumps(use["input"])})
            items.append({"type": "function_call_output", "call_id": call_id, "output": use["answer"]})
        items.append({"role": r["role"], "content": r["text"]})
    return items


for label, replay in (("plain", plain_replay(ROWS)), ("with calls", replay_with_calls(ROWS))):
    print(f"--- {label}: {len(replay)} items")
    for item in replay:
        kind = item.get("type") or item["role"]
        print(f"  {kind:22} {item.get('content') or item.get('name') or item.get('output')}")

# In the plain replay, "Closed #12." and "I'll close #13 now." look alike: two
# sentences about closing. Only the replay with calls shows that one reply did
# something and the other only said it would.

Exercise

Take an agent of yours that calls tools across several turns. Dump the exact request it sends on its fifth turn and mark every earlier tool call: is it there as a call, as text, or missing? Then freeze a history at a point where the agent should act, run the next turn twenty times with your current replay and twenty times with earlier calls replayed in the provider's own call format, and count the turns that actually call the tool.

Hint

Use the provider's own shape for earlier calls: an assistant item carrying the call and a following item carrying its result, linked by the same id. Derive those ids from your stored message id so they never change between turns, or you will trade a behavior fix for a cache miss on every turn.

Saying Is Not Doing

Measured Before It Was Fixed

Each Vessel, Its Own Shape

The Details That Keep It Safe

Code

External links

Exercise

Progress

Comments 0