Skip to content
C.W.K.
Stream
Lesson 03 of 04 · published

The Codex Queue and the Turns That Lie

~16 min · codex, json-rpc, eventual-consistency, idempotency, adapters

Level 0Loose Note
0 XP0/36 lessons0/12 achievements
0/100 XP to next level100 XP to go0% complete

A Second Door, Built Differently

Codex has no peer inbox like Claude's. What it has is an App Server that speaks JSON-RPC, and for local clients that conversation runs as WebSocket messages over a Unix socket. The Codex Desktop app runs its own App Server for its own threads. Coop does not touch that one. It talks to a separate observer daemon, started with codex app-server daemon start, whose private socket it checks with the same private_socket test the Claude door uses.

Why a separate daemon, and why does that matter? Because Coop's first native probe found that the default daemon can read a Desktop thread without owning it, and that resuming the thread there would create a second writer for a chat Desktop is driving. So CodexDesktop is fenced three ways. Its request() refuses any method outside a small list (thread/queue/add, thread/items/list, thread/turns/list plus read-only discovery), and refuses any call whose threadId is not the enrolled thread. Its verify() refuses a thread Codex has archived, re-checks the socket, and refuses outright if the daemon has the thread loaded: Desktop adapter cannot substitute an independent loaded writer. Coop observes Desktop's thread; it never becomes its driver.

Queue, Then Insist on the Echo

Delivery is one call: thread/queue/add with the thread id, the prompt as a text input, and clientUserMessageId set to the outbox message id, the same id the Claude door puts in its frame's uuid. Codex answers with a queued submission. If that submission's clientUserMessageId is not the id Coop sent, the adapter refuses: Native queue accepted a different client identity. An echo that does not match is not a receipt for this message.

Two warnings came out of the first probes. A queued input becomes a native userMessage, the same role a human's typing has, not a peer envelope with provenance. So the prompt itself opens with Coop's provenance line, and the architecture forbids ever reading that native role as Dad's authorization. And the queue is not idempotent. Two immediate adds with the same clientUserMessageId returned two different queue ids; an add after consumption returned a third. A queue id is a receipt, not an exactly-once guarantee. That is why the previous lesson's rule is not optional here: the engine flips the row to uncertain and the host queue records the attempt before the call, and recovery observes by client id instead of sending again. Read the duplicate check exactly, because it is narrower than it sounds. Once a pass has recorded which turn took the input, a newer turn carrying the same client id is a Conflict (Duplicate native queue consumption), never a guess about which one counts, and so is a second match among the items searched for a steered input. But the summary walk goes newest first and stops at the first turn it finds with the id, so two copies already sitting in the history on the first read come back as one receipt, for the newer copy. The observer has no check for that case; what prevents it sits upstream, in the rule that recovery observes and never sends again.

Two Lists, Two Clocks

The receipt and the worker's answer both live in the thread's history, and Codex offers two ways to page through it. On 2026-09-30 a Haven worker lost two invitations and two reports to the difference between them.

thread/turns/list answers from the rollout plus the live turn state. thread/items/list answers from a thread store that Codex materializes from the rollout afterward. So a turn can read as completed while its final message is not yet listable as an item, and it stays that way for minutes. The first observer read each finished turn's items once, saw no final, closed the turn, and never looked again. The COOP_ACCEPT and the report were both real; they had simply not arrived in the slower list yet.

The fix was to stop asking two sources a question that only one should answer. Finals now come only from turn summaries (itemsView: summary): one response, from one source, carrying a turn's status, its first input with its clientId, and the final Codex itself selects. The item store is searched for exactly one case: an input steered into an already running turn, which is not that turn's first input and so is not in any summary. Even then the search starts from an item horizon recorded at send time, so it never rereads older history. Without a horizon, one delivery once cost 133 requests and 55 MB.

"Interrupted" Does Not Mean Over

The second trap was in the status field. The observer daemon does not run Desktop's turns, and it labels every turn it does not see running as interrupted. So a turn Desktop is in the middle of reads interrupted, with no completedAt. Every stall of 2026-09-30 (two invitations and two reports, Haven tasks #270 and #268) was a running turn taken as ended.

The rule now is structural, not lexical: a turn has ended only when it carries completedAt, or when a later turn has begun, because turns run one at a time. The status string is consulted only after that test passes. The adapter's activity reading uses the same rule, measured on office on 2026-10-08: a working Desktop thread's newest turn read interrupted with no completedAt; an idle one read completed with it.

The same history taught one more bound. A single thread measured more than 27 MB, with one item alone at 1.05 MB, over the 1 MB WebSocket frame limit the first client used. The adapter now reads ten-item pages and accepts a large single message, so one oversized item cannot blind it.

When the Daemon Is Not There

An observer that needs a daemon has a new failure: the daemon is gone. Since 2026-10-05 (commit 89420c4) the adapter recovers it, conservatively. It rediscovers only when the enrolled socket path no longer exists; a socket that exists but fails the privacy check stays a refusal. Discovery looks for exactly one private daemon socket; two or more is a Conflict that asks for operator reconciliation, because picking one would be a guess. If none exists, it starts the daemon with the idempotent start command, under an exclusive flock on a cooldown file shared by every adapter child and by discovery. Attempts, failed ones included, are spaced at least 30 seconds apart, so twenty adapters noticing the same missing daemon produce one start, not twenty. The start's PID and version are logged, and no running native work is restarted. While it is still missing, the message says exactly that: Codex observer daemon is not running. Coop tried to start it: {error}. Coop will retry shortly.

A status field is another process's opinion; structure is evidence. "Completed" from a list that materializes later, and "interrupted" from a process that does not run the turn, were both honest reports of the wrong thing. Ask the one source that sees the whole fact at once, and decide on what cannot be mislabeled: a timestamp that exists, a later turn that began, an id that echoes back.

Code

desktop.py: one thread, one queue call, one echo (verbatim excerpt)·python
    async def request(self,method,params):
        if method not in ('initialize','thread/loaded/list') and params.get('threadId')!=self.session_id:
            raise Conflict('Codex adapter cannot target a different session')
        return await super().request(method,params)

    async def send(self,message_id,prompt,checkpoint):
        await self.verify()
        result=await self.request('thread/queue/add',{'threadId':self.session_id,'clientUserMessageId':message_id,'input':[{'type':'text','text':prompt}]})
        queued=result.get('queuedSubmission',{})
        if queued.get('clientUserMessageId')!=message_id:
            raise Conflict('Native queue accepted a different client identity')
        return {'queue_id':queued['id']}
desktop.py observe_work: when a turn has ended, oldest first (verbatim excerpt)·python
        finals=[];in_order=True
        ascending=newer[::-1]
        for index,turn in enumerate(ascending):
            begun_after=index<len(ascending)-1
            if not begun_after and not (turn['status'] in self.TERMINAL and turn.get('completedAt') is not None):
                break  # still running in Desktop, however the daemon labels it
            final=next((i for i in reversed(turn.get('items',[])) if i.get('type')=='agentMessage'),None)
            if turn['status']=='completed' and final and final['id'] not in seen:
                finals.append({'session_id':self.session_id,'native_message_id':final['id'],
                               'native_turn_id':turn['id'],'text':final.get('text',''),
                               'completed':True,'native_peer_role':False})
                seen.add(final['id'])
codex_daemon.py: one shared cooldown for every caller (faithfully trimmed excerpt)·python
START_INTERVAL = 30

    def ensure_socket(self):
        found = self.sockets()          # more than one private socket is a Conflict
        if found:
            return str(found[0])
        self.state.parent.mkdir(parents=True, exist_ok=True)
        fd = os.open(self.state, os.O_RDWR | os.O_CREAT | os.O_NOFOLLOW, 0o600)
        with os.fdopen(fd, 'r+') as handle:
            # All adapter children and discovery share this cooldown, including
            # failed attempts. Only the idempotent start command is permitted.
            fcntl.flock(handle, fcntl.LOCK_EX)
            found = self.sockets()
            if found:
                return str(found[0])
            ...
            if now - previous.get('attempted', 0) < START_INTERVAL:
                raise Conflict(self.unavailable(previous.get('error', 'Waiting for the previous start attempt')))
            ...
                result = subprocess.run([str(self.binary()), 'app-server', 'daemon', 'start'],
                                        capture_output=True, text=True, timeout=15)

External links

Exercise

Find an API you depend on that exposes the same data through two endpoints, such as a list endpoint and a detail endpoint, or a status field and an event log. Write a small script that creates something and then polls both endpoints every second, logging when each first shows the new state. Measure the gap. Then look at your own code: is any decision made by reading one endpoint for status and the other for content?
Hint
Search engines, object stores, analytics APIs and anything with a cache or a materialized view are good candidates. If the gap is ever nonzero, a decision that joins the two sources is a race. The cure is the one Coop used: take the status and the content from the same response, and treat a later structural fact, like a newer record existing, as stronger than any status label.

Progress

Progress is local-only — sign in to sync across devices.
Spotted a bug or have feedback on this page?Report an Issue

Comments 0

🔔 Reply notifications (sign in)
Sign in — Please sign in to comment.

No comments yet — be the first.