Skip to content
C.W.K.
Stream
Lesson 01 of 04 · published

Notifications Are Rows Without a Session

~14 min · outbox, notifications, transactions, backoff, durability

Level 0Loose Note
0 XP0/36 lessons0/12 achievements
0/100 XP to next level100 XP to go0% complete

Two Kinds of Mail in One Outbox

Everything Coop sends goes through one table, engine_outbox. Each row has a kind, a JSON payload, a state, an attempts counter, a due time, and one column that decides who the row is for: session_id.

  • A row with a session is mail for a worker. Invitations, work envelopes, reply reviews, follow-ups and notices all carry the exact session (and generation) that must receive them, and that session's adapter is the only thing that delivers them. Track 4 followed those through the native doors.
  • A row with no session is mail for Dad. There is no native receiver to hand it to, so it waits for a different reader.

The architecture states the second kind as a list: an outcome a worker or the chief recorded, a blocked report, a declined or expired invitation, a lost worker session, a chief escalation or park. In code those are two kinds. outcome rows carry done, released or abandoned. attention rows carry a reason: blocked_report, invitation_declined, invitation_expired, session_disconnected, session_exited, chief_escalation, chief_park, plus a few the assignment flow adds (a failed or expired setup, a remote session that is ready, a CLI take that was adopted, a worker's reply to Dad). There is no separate notifications table and no send_to_dad() anywhere in the domain. Dad's notification is just an outbox row nobody native will ever claim.

Written in the Same Transaction

Look at where an outcome notification is born. When a report closes work, _record_outcome runs inside the engine's command transaction and does five things in a row: insert the report, mark the binding resolved, cancel any pending reply review, resolve the outcome obligation (closing the dispatch if nothing else is bound), and enqueue the outcome row for Dad. One BEGIN IMMEDIATE, one commit.

That ordering is the whole point. The alternative, closing the work and then calling Telegram, has two crash windows. Crash after the close and before the call, and Dad never hears that the work finished. Call first and crash before the commit, and Dad hears about a close that never happened. Writing the notice into the same transaction as the change it announces removes both: either the close and its notice exist together, or neither does. This is the transactional outbox pattern, and Coop's architecture names it plainly: canonical mutations write their outbox entries in the same transaction.

The Notifier: Fifteen Seconds, Ten Rows

The reader is Notifier, a small task in Coop's chief.py. Every 15 seconds it selects session-less rows that are pending and due, oldest first, at most ten. For each it composes Dad's text:

  • A head line: [Coop] <head>: <label>. The head is a short word with an emoji (✅ done, ⏸ worker blocked, ⌛ invitation not accepted, 🅿️ parked for Dad, 👀 chat enrolled). The label is the work item, read fresh from the ledger: todo #N (scope) title.
  • Below it, the human lines. dad_text takes the worker's or chief's for_dad lines when they exist, cut at 1,200 characters (FOR_DAD_LIMIT). Without them it falls back to an excerpt of the English report body, with any pasted report JSON and hidden report lines stripped out. The code comment says it exactly: the English excerpt is the fallback, not the message. Report JSON never reaches Dad, whatever a worker pasted.

Track 5 showed how for_dad gets into a report. This is where it comes out.

Failure Defers; It Does Not Drop

The composed text goes to Pippa's notify door (the next lesson). If Pippa answers, the engine settles the row as delivered with a receipt naming the channel. If anything goes wrong, the row is deferred: attempts goes up by one and due moves to now plus min(300, 2 ** min(attempts, 8)) seconds. The waits double from two seconds; the code writes the ceiling as 300, but the exponent stops at 8, so the longest wait is 256 seconds. With quick failures and a 15-second tick, twelve attempts take about twenty-four minutes, and the twelfth marks the row failed. (Worker-bound rows give up after five.)

Read "a Pippa outage delays a notification and never loses it" with that in mind. A restart or a short outage costs Dad a few minutes. A long one turns the row failed, and it does not vanish. The row stays in the store; for the week after it was created, Coop's Operations view lists it as could-not-be-delivered with the attempt count and the last error, and the workspace badge counts it among the things that need Dad. Nothing deletes it and nothing retries it forever. Bounded and visible, never silent.

One more edge, for completeness. Maintenance expires a session-less row that is still pending three days after it was created, recording stale-before-delivery. With a running notifier that cannot happen to a live row; it exists for the case where no reader ran at all, which is exactly the case Track 6 found.

What Never Becomes a Notification

Just as important is what is not in that list. A session going idle is not news. A repeated observation is not news. A worker's prose final without a report becomes a note for the chief, not a ping. And a native version change is recorded on the session row and never announced. That last one was learned the hard way: the Claude adapter once carried a version allow-list, and as a gate it blocked every assignment; demoted to a note, it still read to Dad like a block. On 2026-10-02 it was removed. Dad hears about decisions, losses and things only he can unblock, not telemetry.

A notification is a row, not a call. Write it in the same transaction as the change it announces, let one reader carry it with a bounded retry, and let a failure end as a visible state rather than a disappearance. Then a notice can still fail to reach Dad, but never without a trace: whatever was not delivered is a row with a state and a reason, not a silence.

Code

cwkCoop chief.py, Notifier (trimmed excerpt: the head table is shortened)·python
def compose(self,row):
    payload=json.loads(row['payload'])
    item=payload.get('item') or {}
    ...  # label = "todo #N (scope) title", read fresh from the ledger
    # Dad reads the worker's or chief's lines in his language when they
    # exist; the English excerpt is the fallback, not the message. Report
    # JSON never reaches him, whatever a worker pasted (todo #360).
    text=dad_text(payload.get('for_dad') or payload.get('text') or '')
    if row['kind']=='outcome':
        head={'done':'✅ done','released':'↩️ released','abandoned':'⛔ abandoned'}.get(payload.get('outcome'),payload.get('outcome',''))
        return f"[Coop] {head}: {label}\n{text}".strip()
    words={'blocked_report':'⏸ worker blocked','invitation_expired':'⌛ invitation not accepted',
           'chief_park':'🅿️ parked for Dad','session_exited':'🔌 worker session exited', ...}
    head=words.get(reason,reason.replace('_',' '))
    return f"[Coop] {head}: {label}\n{text}".strip()

def pending(self):
    with self.engine.connection() as c:
        return [dict(r) for r in c.execute("SELECT * FROM engine_outbox WHERE session_id IS NULL AND state='pending' AND due<=? ORDER BY created LIMIT 10",(time.time(),))]

async def tick(self):
    for row in self.pending():
        try:
            text=self.compose(row)
            await self.client.call('notify',{'text':text,'kind':row['kind'],'outbox_id':row['id']})
        except Exception as exc:
            self.engine.delivery('defer:%s:%s'%(row['id'],row['attempts']),row['id'],None,None,'defer',{'last_error':f'{type(exc).__name__}: {exc}'[:300]},self.meta)
            continue
        self.engine.delivery('notified:'+row['id'],row['id'],None,None,'notified',{'channel':'pippa-notify','text':text[:300]},self.meta)
A minimal sketch: the defer schedule of one notification while Pippa is down·python
# Mirrors Engine.delivery('defer'): attempts + 1, due = now + min(300, 2 ** min(attempts, 8)),
# and state 'failed' at the twelfth attempt. The Notifier only looks every 15 seconds.
import math

TICK, LIMIT = 15, 12
t, attempts = 0, 0
while True:
    attempts += 1                      # this tick's handoff to Pippa failed
    if attempts >= LIMIT:
        print(f"attempt {attempts:2d} at {t:5d}s -> failed, still listed for Dad")
        break
    wait = min(300, 2 ** min(attempts, 8))
    t_next = math.ceil((t + wait) / TICK) * TICK   # first Notifier tick at or after due
    print(f"attempt {attempts:2d} at {t:5d}s -> due again in {wait:3d}s, picked up at {t_next}s")
    t = t_next

# Last lines of output:
# attempt 11 at  1140s -> due again in 256s, picked up at 1410s
# attempt 12 at  1410s -> failed, still listed for Dad

External links

Exercise

Find one place in your own project where code commits a change and then sends a message about it inline: an email after signup, a Slack post after a deploy, a webhook after an order. Rewrite it so the commit inserts an outbox row in the same transaction, and a separate loop reads pending rows, sends them, and on failure pushes due forward with a capped doubling backoff and a maximum attempt count. Then turn off the network for ten minutes and watch two numbers: the age of the oldest pending row and the count of failed rows.
Hint
The first design question is not the backoff curve, it is the classifier: which column tells the reader a row is its to deliver? Coop uses session_id IS NULL. Pick something equally boring, and make the failed state something a human screen actually shows.

Progress

Progress is local-only — sign in to sync across devices.
Spotted a bug or have feedback on this page?Report an Issue

Comments 0

🔔 Reply notifications (sign in)
Sign in — Please sign in to comment.

No comments yet — be the first.