Skip to content
C.W.K.
Stream
Lesson 03 of 05 · published

A Prose Final Becomes a Note

~13 min · observability, idempotency, evidence, reporting

Level 0Loose Note
0 XP0/36 lessons0/12 achievements
0/100 XP to next level100 XP to go0% complete

What Workers Actually Say

The previous two lessons describe what a worker should end with. Here is what workers often end with: shipped, pushed, tests green. Plain prose, no JSON, no binding id. Sometimes the worker forgot the format. Sometimes it is mid-task and just finished a turn. Sometimes a person was talking to it and it answered them.

A strict system has two bad options here. It can ignore the final because it is not a report, which throws away exactly the sentence Dad would have read before deciding the work was finished. Or it can try to infer an outcome from the prose, which turns a model's casual phrasing into a ledger transition nobody authorized. Coop takes neither. Since commit 4a99306 (2026-09-30, the day the chief became Dad's proxy), a final without a structured report becomes a note.

The Note

In Coordinator.observe_work, each completed final from the exact native session is checked for a report. If it carries none and the envelope still has active bindings, the adapter calls Engine.note with the final's text. The comment beside that call is the whole rationale: the worker's own words are the chief's evidence when no structured report came, because shipped and pushed in prose is what Dad would read before closing.

A note is deliberately small. The engine keeps text.strip()[:NOTE_LIMIT], and NOTE_LIMIT is 4,000 characters. The schema calls the table a bounded excerpt per native message, never a transcript. Notes are not canonical state: they stay out of the change feed and are pruned after thirty days. They are keyed by the native message id with INSERT OR IGNORE, so seeing the same final twice stores it once.

Who reads them? The chief. When it reviews an obligation, its context includes the six most recent notes on that dispatch next to the reports, deliveries and events. It reads them the way Dad would read the chat. Notes also decide whether there is anything to review at all: when a worker goes idle on an obligation the chief has already reviewed, the engine pulls it forward only if a note newer than that review exists. An idle worker that said nothing new is not news.

A Rejected Report Is Also a Note

The second use arrived with commit 4e78f1e (2026-10-05). Before it, if Engine.report refused a report (a done with an unanswered reply, a stale revision), the exception escaped the observation pass before the cursor moved. On the next pass the adapter read the same final, tried the same report, and failed the same way. Forever. The worker could send a corrected report in its next turn and the adapter would never reach it, because it was stuck replaying the old one. A poison message, in queue terms.

Now the adapter catches the refusal and records a note instead: Report rejected: followed by the engine's reason and the final's text. Then it moves on. The rejection is evidence the chief can read, the obligation is still owed, and the worker's next report gets its turn. The architecture puts it in one line: rejected native reports remain notes for review; they never hide a later corrected report.

Read that line as a promise about what the engine refuses, because that is all the catch covers. Two checks run in the adapter before the engine is ever asked, outside the try: an entry in coop_reports that is not a JSON object raises Invalid work report, and a second report for the same binding, or one carrying a key outside REPORT_FIELDS, raises Duplicate or malformed per-item report. Those escape the pass exactly the way engine refusals used to. The cursor stays where it was, the next pass reads the same final and fails the same way, and the worker's corrected report waits behind it. The poison message is cured for engine refusals and still alive for malformed entries.

The Cursor Moves Last

All of this rests on an ordering rule that is older than notes (commit 2f9c779, 2026-09-28): the adapter persists its transcript cursor only after every effect from that page of finals has committed. If the host crashes halfway, the next pass starts from the old cursor and replays finals it already handled. That is safe because every effect is keyed: a report is an engine command with an idempotency key built from the envelope, the native message id and the binding, and a note is unique per native message. Replay is idempotent; skipping is impossible.

Reverse the order and you get the opposite failure. Advance the cursor first, crash before the effect, and that final is never read again. A lost report is far worse than a repeated one, because nothing downstream even knows to look for it.

Three Facts, Not One

Watch how many different things observe_work records. The native harness acknowledging that the work envelope arrived is a delivery receipt. A completed assistant turn is a final, and becomes a note or a report. A committed done, released or abandoned is a terminal work report. The architecture insists these are three separate facts. Receiving the task is not starting it, finishing a turn is not finishing the work, and an ordinary final or a blocked report leaves observation running for the next turn.

Keep what the worker said, never decide from it. A note preserves the evidence without promoting it into a transition. The decision stays with whoever holds the authority to make it, and that reader gets the same words Dad would have seen.

Code

Coordinator.observe_work, trimmed: notes, rejected reports, and the cursor last·python
for final in page['finals']:
    reports=report_text(final['text'])
    if not reports and active and hasattr(self.engine,'note'):
        # The worker's own words are the chief's evidence when no
        # structured report came: "shipped and pushed" in prose is
        # what Dad would read before closing. Bounded, idempotent.
        self.engine.note('note:'+final['native_message_id'],sid,generation,payload['dispatch_id'],
                         final['native_message_id'],final['text'],self.meta)
    for report in reports:
        # Adapter-side checks, outside the try below: these still escape the pass.
        if not isinstance(report,dict):
            raise ValueError('Invalid work report')
        bid=report.get('binding_id')
        ...
        if bid in seen or set(report)-REPORT_FIELDS:
            raise Conflict('Duplicate or malformed per-item report')
        ...
        try:
            result=self.engine.report('native-report:'+oid+':'+final['native_message_id']+':'+bid, ...)
        except (Conflict, ValueError) as exc:
            # A rejected done is evidence, not a permanently replayed
            # poison final that hides the worker's next corrected report.
            self.engine.note('rejected-report:'+final['native_message_id']+':'+bid, ...,
                'Report rejected: '+str(exc)+'\n'+final['text'],self.meta)
            continue
        ...
# Advancing the cursor last makes crash replay idempotent. A plain
# final or a blocked report never stops observing future native turns.
self.queue.progress(oid,page['checkpoint'],not active)
A minimal sketch: keyed effects first, cursor last, and a crash that replays safely·python
import sqlite3

db = sqlite3.connect(":memory:")
db.execute("CREATE TABLE notes (message_id TEXT PRIMARY KEY, text TEXT)")
db.execute("CREATE TABLE cursor (id INTEGER PRIMARY KEY CHECK (id = 1), position INTEGER)")
db.execute("INSERT INTO cursor VALUES (1, 0)")

transcript = [("m1", "Shipped and pushed."), ("m2", "Tests green; waiting on the build.")]
NOTE_LIMIT = 4000

def observe(crash_after=None):
    (position,) = db.execute("SELECT position FROM cursor").fetchone()
    for index, (message_id, text) in enumerate(transcript[position:], start=position):
        # Effects first, keyed by the native message id: a replay is a no-op.
        db.execute("INSERT OR IGNORE INTO notes VALUES (?, ?)", (message_id, text.strip()[:NOTE_LIMIT]))
        db.commit()
        if crash_after == message_id:
            raise SystemExit("crashed before the cursor moved")
    # The cursor moves last, after every effect of this page is committed.
    db.execute("UPDATE cursor SET position = ?", (len(transcript),))
    db.commit()

try:
    observe(crash_after="m2")
except SystemExit as stop:
    print(stop)
observe()   # replays m1 and m2 from the old cursor
print(db.execute("SELECT count(*) FROM notes").fetchone()[0], "notes")  # 2, not 4

External links

Exercise

Find a consumer in your own work that reads a stream with a cursor: a log tailer, a webhook processor, a job that walks new rows. Check two things. Does it advance the cursor before or after its effects commit? And what happens to an input it cannot parse or that its handler rejects? Kill it mid-page and feed it one bad input, and write down what you lost.
Hint
Look for the line that saves the position. If it sits above the effect, a crash loses data; if an exception can skip it, one bad item freezes the stream. Keying each effect by the input's own id is what makes moving the cursor last safe.

Progress

Progress is local-only — sign in to sync across devices.
Spotted a bug or have feedback on this page?Report an Issue

Comments 0

🔔 Reply notifications (sign in)
Sign in — Please sign in to comment.

No comments yet — be the first.