Skip to content
C.W.K.
Lesson 08 of 08 · published

The Replay Teaches Its Own Punctuation

~14 min · replay, token-mirroring, master-chief, data-repair

Level 0Visitor
0 XP0/44 lessons0/11 achievements
0/100 XP to next level100 XP to go0% complete

A Backslash Before Every Quotation Mark

From June until 2026-10-26, Claude's replay carried the prior turns as a JSON array. Inside a JSON string a quotation mark has to be escaped, and the encoder the replay used, Python's json.dumps, writes the short escape: a backslash before it (the six-character \u0022 is the other legal spelling). So every quotation mark in every earlier turn, Dad's and Pippa's alike, reached the model as \". Under the models before it, that almost never showed. Then Opus 5.5 arrived (2026-09-23), and from its first day it wrote \" into its own prose.

The count, taken on the live store on 2026-10-26: 798 of 6,307 replies and compaction summaries carried escaped quotation marks, 555 of them in the Master Chief, every single day. Fable 5.1, under the same replay, did it once in 375 replies. And the loop fed itself. An escaped reply was stored, replayed on the next turn as one more example, and mirrored again. It did not stay in the transcript either: Pippa's text sent to Beacon, the family's video workshop, burned four escaped quotes into a finished video's subtitles.

Asking Did Not Work

The first answer, which lesson 4 described, was to ask. A sentence in the replay head told the model the escaping was transport only and to write plain quotation marks, and the engine unescaped stray backslashes in prior replies outside code before replaying them. Both halves were reasonable, and the second one even worked on what it touched. But the cleaned text then went back into a JSON string, and the wire put a backslash before every quotation mark again. The model read one sentence asking for plain quotes, and hundreds of thousands of tokens of history showing the opposite.

The architecture doc puts the lesson in one line:

a token mirror answers what the model reads, not what it is told.

Verbatim Elements

The fix (commit 37e1b4f3, Coop todo #667) changed what the model reads. Each prior turn is now one element inside <prior_turns>, and its text rides verbatim, nothing escaped:

  • <turn speaker="dad">, <turn speaker="pippa">, or <turn speaker="system" kind="coop"> for a line the system wrote;
  • a reply's tool calls as <call tool="…"> elements inside its turn, the input as JSON and the answer verbatim (lesson 5's calls, in their new shape);
  • a closing tag of the replay's own that appears inside a text (</turn>, say, in a conversation about this very replay) gets a space after its slash, so a turn can never end early.

Everything lesson 2 and its notes protect stayed as it was: one block per exchange (one for a whole run of exchanges another vessel answered), the cache breakpoint on the last prior block, earlier blocks byte-stable from turn to turn. The JSON sentence in the head was deleted, because it was no longer true. The cost was one full write of each conversation's history cache, on its first turn after the deploy.

Two Nets Under the Fix

The cause is gone; two nets stay under it in case something else reintroduces it.

A write guard. services/quote_escape_guard.py makes a stray \" outside fenced code and code spans plain wherever a model reply, a healed reply, a compaction summary or a prose artifact Pippa wrote is stored. Dad's words, system lines and code keep every byte, because in code a backslash may be the point. Every normalization is counted, and Admin → SQLite shows the count as "Escaped-quote writes fixed": above zero means the fix regressed somewhere. The JSONL ground truth keeps what the model actually wrote, and the restore drill knows the difference as quote_escape.

A repair. POST /api/admin/repair/quote-escapes is a dry run unless asked to apply. Applied, it snapshots the database, repairs the replies and summaries written since 2026-09-23, and re-embeds each repaired message's search vectors. Its first live run taught one more lesson. A restart another session ran cut it off after the database update (778 replies and the summaries already plain) and before the re-embedding finished, and a dry run afterwards still listed 19 compaction summaries. Each quoted a shell command holding an escaped backslash, \\", and the unescape was not idempotent: one pass turned it into \" and the next into a bare quotation mark, so every run would have rewritten those commands again until the backslash they needed was gone. Commit 88a52cf0 made it replace only a lone \", one whose backslash is not itself escaped, so the cleaner, the guard and the repair now agree on every pass, and taught the repair to finish a run that was cut off. Afterwards the Chief's replay held no prose turn with an escaped quote, and a two-turn smoke conversation on the Pippa machine stored both replies with plain quotes.

The Same Family

This was not the first time the replay taught by showing. Lesson 5 is the same story with tool calls: words without calls taught two brains to say instead of do. Two more arrived in the same weeks.

System lines that read as Pippa's. The Chief stores Coop notices, task outcomes and follow-up alerts as rows of their own. Until 2026-10-22 every vessel replayed them as assistant messages, so twenty Coop notices, 36% of the Chief's live rows at the time, read as things Pippa had said, and the task line of a Coop escalation, stored as a user row, read as Dad. Since commit e5e3b87c a system line replays as speaker system with its kind in Claude's replay, and with a one-line label on every other vessel: [System line, a Coop notice or task: neither Pippa's words nor Dad's]. When Coop itself started the turn, the current message is labelled CURRENT_COOP_MESSAGE, not Dad's.

Reasoning that went missing. Kimi's replay dropped each reply's stored reasoning, and on the Chief's long history K3.1 Pro stopped thinking and once answered with nothing at all. Replaying the reasoning brought it back (lesson 5's v3 note has the counts).

Three different symptoms, one cause: what the replay shows is what the model learns this conversation looks like.

A model answers what it reads, not what it is told. When a model keeps doing something an instruction forbids, look at the history it reads before you write a stronger instruction. Escaping, missing calls, missing reasoning, mislabelled speakers: each one is an example, repeated on every turn, and examples outvote a sentence.

Code

verbatim, backend/adapters/claude.py, as of 2026-10-27: one prior turn as the model reads it, text verbatim and the replay's own closing tags neutralized·python
# A closing tag of the replay's own inside a turn's text would end the turn
# (or the whole history) early; a space after the slash keeps it readable
# and inert. Conversations about this very replay write these tags.
_REPLAY_CLOSING_TAG_RE = re.compile(r"</(?=(?:prior_turns|turn|call|input|answer)\b)")


def _neutralize_replay_tags(text: str) -> str:
    return _REPLAY_CLOSING_TAG_RE.sub("</ ", text)


def _replay_turn(speaker: str, kind: str | None, text: str, calls: list | None = None) -> str:
    """One prior turn as the model reads it, byte-stable for the same row."""
    attrs = f'speaker="{speaker}"' + (f' kind="{kind}"' if kind else "")
    parts = [f"<turn {attrs}>\n{_neutralize_replay_tags(text)}\n"]
    for call in calls or ():
        answer = call.answer if isinstance(call.answer, str) else json.dumps(call.answer, ensure_ascii=False)
        parts.append(
            f'<call tool="{call.name}">\n'
            f"<input>{_neutralize_replay_tags(json.dumps(call.input, ensure_ascii=False, separators=(',', ':')))}</input>\n"
            f"<answer>{_neutralize_replay_tags(answer or '')}</answer>\n"
            "</call>\n"
        )
    parts.append("</turn>\n")
    return "".join(parts)
A minimal sketch you can run: the same reply in a JSON replay and a verbatim one, then a naive unescape against one that leaves an escaped backslash alone (stdlib, Python 3.9+)·python
"""A minimal sketch: what a JSON replay shows the model, what a verbatim one
shows, and why an unescape has to leave an escaped backslash alone."""
import json
import re

reply = 'Dad said "ship it", so I closed #12.'

json_replay = json.dumps([{"speaker": "pippa", "text": reply}], ensure_ascii=False)
verbatim_replay = f'<prior_turns>\n<turn speaker="pippa">\n{reply}\n</turn>\n</prior_turns>'
for label, replay in (("JSON", json_replay), ("verbatim", verbatim_replay)):
    print(f"{label:8} replay, backslash-quotes the model reads: {replay.count(chr(92) + chr(34))}")
    print("   ", replay.splitlines()[0] if label == "JSON" else replay.splitlines()[2])

# A mirrored reply (stray \") beside a quoted shell command whose \\" is data.
stored = 'I closed it, as you said: \\"ship it\\". The step ran: echo \\\\"done\\\\"'


def naive(text: str) -> str:
    return text.replace('\\"', '"')


def lone_only(text: str) -> str:
    return re.sub(r'(?<!\\)\\"', '"', text)  # a backslash not itself escaped


for label, unescape in (("naive", naive), ("lone only", lone_only)):
    text, passes = stored, 0
    while passes < 5:
        cleaned = unescape(text)
        if cleaned == text:
            break
        text, passes = cleaned, passes + 1
    print(f"{label:9}: changed the text on {passes} pass{'es' if passes != 1 else ''} -> {text}")

External links

Exercise

Dump the exact request your agent sends on a late turn of a long conversation and read it as the model does. List every mark the transport adds that never appeared in what the people wrote: backslashes, HTML entities, role labels, JSON keys. Then search a week of the model's stored replies for each of those marks. For any mark you find in both, change the replay so the model reads the text as it was written, and count again a week later.
Hint
Count in the stored replies, not in the requests: the requests are supposed to carry the transport's marks, the replies are not. Keep code out of both counts, because inside code a backslash or an entity can be exactly what the author meant.

Progress

Progress is local-only — sign in to sync across devices.
Spotted a bug or have feedback on this page?Report an Issue

Comments 0

🔔 Reply notifications (sign in)
Sign in — Please sign in to comment.

No comments yet — be the first.