Budget, Rotation and the Silent Outbox

A Reviewer Needs Walls

A chief that wakes on evidence and can close work is powerful, which means it needs limits that do not depend on its own judgement. Every one of them lives in Chief.prepare, the function that decides whether a review starts at all, and in the runner around it.

One active run per role. A unique partial index on engine_chief_runs allows a single run in prepared, running or uncertain state for the role. While one is in flight, prepare hands back that run rather than starting another.
An hourly budget of 12. If twelve runs were created in the last hour, no new run starts. Nothing is dropped: the obligations stay pending and visible, and the next tick after the window slides can start one.
At most 50 obligations per run, most overdue first. The wake door on Pippa's side refuses a longer list too.
Unmentioned means fifteen minutes, not now. If the chief's answer skips an unparked obligation it was given, that obligation is re-reviewed in 900 seconds (UNMENTIONED_SECONDS), so a forgetful answer becomes a bounded follow-up instead of a hot loop. A skipped parked obligation is the exception: its due time stays where it was, already in the past, so the very next run offers it again and only the hourly budget bounds that loop.
A failed wake retries in fifteen minutes. If the turn fails, is superseded, or returns decisions that break the contract, the run is marked failed and its obligations come due again in 900 seconds (FAILED_SECONDS). The runner itself catches every exception and keeps ticking every five seconds, because a stopped runner would leave every obligation unreviewed while the service still reported ready.
Dad is asked once. A repeated park or escalation on the same bound obligation, with no new report, worker note or message since the last one, is recorded but not sent.

One more wall was learned the hard way. The 2026-09-30 review found that 458 of 1,288 rows in engine_commands were prepare results: the runner asked every five seconds while a turn was running or the budget was spent, and each ask left an idempotency receipt. prepare now decides from a plain read and writes a command only when a run actually starts.

Rotation: Full Replay Grows Every Turn

Pippa replays a conversation in full on every turn. That is a fine property for a human chat and a slow leak for a chief that is woken dozens of times a day: by the third production day its conversation held 152 messages, and every wake paid to read all of them again. So the runner rotates. When prepare starts no review (nothing is due, or the hourly budget is spent with obligations still overdue) and the role has reached ROTATE_AFTER, forty wakes, it asks Pippa's bind door for a fresh thread and binds the role to it with the same brain, authority and identity, at the next revision. Runs still in flight under the old revision are superseded. The binder keeps the same context key, so Dad's Sidekick simply opens the new conversation.

Rotation is cheap for a reason worth noticing: none of the chief's state lives in its conversation. Pending obligations outlive a role replacement; every wake reads the current contract and records from the engine; dispositions are rows. The conversation is a workspace, not a memory. That is Invariant 5 again, seen from the other side.

The Outbox Nobody Read

Here is the finding that should stay with you longest. Coop writes its outgoing messages in the same transaction as the change that causes them, exactly as a durable system should. An outcome report enqueues an outcome row; a blocked report, a lost session or a chief escalation enqueues an attention row. Rows with a session go to a worker's adapter. Rows with no session are Dad's notifications.

For the first three days, nothing read that second kind. The assignment coordinator consumed invitation and work rows, and no process anywhere consumed the rest. The review counted 15 outcome rows pending for ever. Every write succeeded. Every row was correct. No error was logged, because nothing was failing; there was simply nobody at the other end. From Dad's chair, delegated work finished in silence.

The repair added a Notifier that composes each session-less row into one line and hands it to Pippa's notify door, deferring failures with a bounded backoff (Track 7 walks that road). And it did one more careful thing: on the first start, rows queued before any notifier existed were expired as history instead of arriving in Dad's chat as a burst.

Measure Effect, Not Activity

Put the three numbers side by side and they tell one story. 74 wakes, every one deferring or escalating, not one obligation affected: the chief looked busy and changed nothing, which is why its authority was widened. 15 notifications written perfectly and delivered never. 458 command rows recording a question nobody needed answered. In each case, the activity metric (wakes, rows written, commands recorded) was healthy, and the effect metric (obligations resolved, notifications delivered) was zero.

The architecture names the instruments it wants instead: obligation age and reason, closure latency, delivery retries, unknown outcomes and Dad's repair interventions, with targets set from measured baselines rather than invented success rates.

A queue nobody drains is a silent failure. Writing to it always succeeds, so it never errors; it only ages. Watch the age of the oldest pending row per kind, not the number of rows written. And measure a reviewer by how often its decisions change state, not by how often it wakes.

Code

Excerpt from Coop's Chief.prepare (trimmed): decide from a read; write only when a run starts·python

def prepare(self):
    """The review to run now, without writing anything unless a new one starts."""
    with self.engine.connection() as c:
        role=c.execute("SELECT * FROM engine_chief_roles WHERE id='main' AND enabled=1").fetchone()
        if not role:
            return None
        active=c.execute("SELECT * FROM engine_chief_runs WHERE role_id='main' AND status IN ('prepared','running','uncertain')").fetchone()
        if active:
            return dict(active)
        now=time.time()
        if not c.execute("SELECT 1 FROM engine_obligations WHERE status='pending' AND due<=? LIMIT 1",(now,)).fetchone():
            return None
        if c.execute("SELECT count(*) FROM engine_chief_runs WHERE created>?",(now-3600,)).fetchone()[0]>=HOURLY_BUDGET:
            return None
    def run(c):
        ...
        due=[r['id'] for r in c.execute("SELECT id FROM engine_obligations WHERE status='pending' AND due<=? ORDER BY due,id LIMIT 50",(time.time(),))]
        ...
        c.execute("UPDATE engine_chief_roles SET wakes=wakes+1 WHERE id='main'")
        return self.engine.row(c,'engine_chief_runs',rid)
    return self.command(str(uuid4()),'chief.prepare',{},run)

Excerpt from Coop's ChiefRunner (trimmed): rotation happens only on a tick where prepare starts no review·python

async def rotate(self,role):
    """Start a fresh chief conversation after ROTATE_AFTER wakes.

    Pippa replays a conversation in full on every turn, so a chief
    conversation that only ever grows makes each wake cost more than the
    last (152 messages after three days). The binder keeps the same
    context key, so Dad's Sidekick opens the new conversation.
    """
    ...

async def tick(self):
    review=self.chief.prepare()
    if review is None:
        role=self.chief.context()['role']
        if role and role['enabled'] and role['wakes']>=ROTATE_AFTER and self.client is not None:
            await self.rotate(role)
        return
    ...

A minimal sketch: two health checks that would have caught the review's findings (synthetic rows shaped like them)·python

import json, sqlite3

db = sqlite3.connect(':memory:')
db.executescript("""
CREATE TABLE outbox (id INTEGER PRIMARY KEY, kind TEXT, session_id TEXT,
                     state TEXT DEFAULT 'pending', created REAL);
CREATE TABLE chief_runs (id INTEGER PRIMARY KEY, decisions TEXT);
""")
now = 3 * 86400.0
# Rows with a receiver get drained; rows for nobody (session_id NULL) pile up.
db.executemany('INSERT INTO outbox(kind, session_id, state, created) VALUES (?,?,?,?)',
               [('work', 's1', 'delivered', 100.0), ('invitation', 's2', 'delivered', 200.0)] +
               [('outcome', None, 'pending', 3600.0 * h) for h in range(1, 16)])
db.executemany('INSERT INTO chief_runs(decisions) VALUES (?)',
               [(json.dumps([{'action': 'defer'}, {'action': 'escalate'}]),)] * 74)

# 1. A queue is healthy when its oldest pending row is young, not when rows exist.
for kind, n, oldest in db.execute(
        "SELECT kind, count(*), min(created) FROM outbox WHERE state='pending' "
        "GROUP BY kind ORDER BY min(created)"):
    print(f'{kind:10} pending={n:3}  oldest={(now - oldest) / 3600:5.1f} h')

# 2. A reviewer is useful when its decisions change work, not when it wakes.
#    defer, escalate and park only reschedule the next look.
EFFECT = {'close', 'release', 'abandon', 'adopt', 'assign', 'retire', 'resend'}
runs = [json.loads(r[0]) for r in db.execute('SELECT decisions FROM chief_runs')]
changed = sum(any(d['action'] in EFFECT for d in run) for run in runs)
print(f'wakes={len(runs)}  wakes that changed anything={changed}')

Exercise

List every queue, outbox, inbox or 'pending' table in a system you run, including the informal ones: a jobs folder, a notifications table, an email drafts label. For each, name the process that drains it and write the one query or command that shows the age of its oldest pending item. Run them. Then pick one automated reviewer or bot you rely on and compute, over its last fifty runs, the share that changed anything.

Hint

If you cannot name the drainer, that is the finding; you do not need to run anything else for that queue. For the reviewer, decide in advance which outcomes count as an effect, the way the sketch separates closing work from merely rescheduling the next look, before you look at the numbers.

A Reviewer Needs Walls

Rotation: Full Replay Grows Every Turn

The Outbox Nobody Read

Measure Effect, Not Activity

Code

External links

Exercise

Progress

Comments 0