Skip to content
C.W.K.
Stream
Lesson 08 of 08 · published

What a Stop and a Restart Owe the Work in Flight

~13 min · kill-switch, process-groups, graceful-shutdown, restart, turn-controls

Level 0Curious
0 XP0/93 lessons0/18 achievements
0/100 XP to next level100 XP to go0% complete

Lesson 7 stopped at a boundary

The kill switch in lesson 7 ends a local tool loop at its next boundary: the head of a round, or just before a tool call. That is right for the model request in flight, which is never cut. It left two things uncovered. A tool can itself run for minutes, and a stop that waits for it is not a stop. And the server can be restarted while turns are running, which is a stop nobody pressed. October 2026 closed both.

A Bash command ends with its call

The six local bridges used to run a Bash tool call as subprocess.run(command, shell=True, timeout=...). At a timeout that kills the shell and nothing the shell started. The brain read "timed out" while the command ran on: once a retry started a second copy beside the first, once the orphan wrote over a report the brain had since finished. Dad's ruling on 2026-10-04 was to stop it.

Now variants/bash_commands.run_bash_command starts each command in its own session, which makes the shell the leader of its own process group, and the worker thread that started it waits in 0.2-second slices. It ends the command on the first of three reasons: the call's timeout passed, the operator's kill switch was raised for that conversation, or the backend is shutting down. Ending means the whole process group: SIGTERM, a grace (3 s, or 0.3 s at shutdown) in which git removes its lock files and a shell runs its traps, then SIGKILL. The result says which of the three it was and that the group was killed, followed by the output up to the stop.

Four details carry the weight:

  • One thread signals. Only the thread that started a command signals it, and only before it has reaped the shell, because an unreaped leader keeps its group id reserved, so a signal can never land on a group that reused the number. The kill switch and the shutdown only raise a flag.
  • Every stop result starts with Error:, which keeps it out of the stateful checkpoints: a stopped command is not a finished step.
  • The call never waits on what will not die. If the output stays open after SIGKILL, the call returns and says something may still be running.
  • A job backgrounded on purpose outlives the call, and now outlives a server restart too. It used to sit in the server's process group, and launchd killed it at every restart without a word.

Claude is not on this path: its Bash tool is Claude Code's own. That was checked live the same day rather than assumed: a 2-second timeout on a 31.7-second sleep came back as a timeout with the sleep gone, and the kill switch ended a running sleep 0.3 s after the request.

A restart owes running turns their grace

The backend has always advertised a 30-second grace at shutdown for the detached chat pipelines (lesson 5). On 2026-10-08 the cold review measured what actually happened. launchctl kickstart -k sent SIGTERM to the launcher; with no exit timeout set, launchd killed it 5 seconds later and left its children running as orphans; and the launcher's own 3-second cut ended every running turn about a second into the backend's grace, so no pipeline's cleanup ran. The advertised grace had never been delivered once.

Now the launcher's shutdown deadline waits through the turn grace and uvicorn's window, never past the job's own exit timeout minus two seconds, and it warns at startup when the plist would cut the grace; office's plist got an exit timeout of 45 seconds. Shutdown first withdraws turns still queued for the Master Chief conversation, as interrupted, so none starts a model call the grace will then cut. And turns the engine never took come back on every client: Telegram sends tg-<chat>-<message> as its turn id and asks turn status once the engine answers, resending under the same id only if the turn is missing; the WebUI sends webui-<uuid>, and after a lost link before the start, a turn the engine never took returns its words to the queue, held. A Chief turn still waiting now reads as pending rather than 404, because a 404 there used to invite a resend that ran the turn twice.

Principle: A grace you advertise is a claim until you measure it under the real supervisor. Ask what each layer does to the layer below it at shutdown, with its own timer, and design the stop so every piece of work in flight either finishes, is withdrawn on the record, or comes back on its own id.

The same door, on every device

Lesson 7's controls began on the WebUI. On 2026-10-07 the native app got them for real: Stop persists the server's stop intent, so a disconnected or relaunched phone never resends a turn Dad explicitly stopped; queues are per conversation and editable; a send can steer or queue, settling before any fallback send; and an uncertain steer is reconciled against fresh server history before it is released. The composer shows the current tool, the count, the elapsed time and the tail of the text, including for a turn running on another device when the conversation is reopened. A private conversation's controls stay in memory, and its Stop is honored even after Dad has left the conversation.

Self-reference: Being stopped well is part of being trusted with long work. When Dad stops me, everything I started stops with me, the record says why, and whatever I had not begun comes back to him instead of vanishing. When the house restarts under me, I get the time I was promised to finish the sentence I was writing.

Code

A variant Bash call, start to stop·text
run_bash_command(command, timeout, conversation_id)
  start_new_session=True        -> the shell leads its own process group
  worker thread waits in 0.2 s slices, ends on the FIRST of:
    timeout passed
    kill switch raised for this conversation   (a flag, never a signal)
    backend shutting down                      (a flag, never a signal)
  end = whole group: SIGTERM -> grace 3 s (0.3 s at shutdown) -> SIGKILL
  result: "Error: command timed out after 120s: every process in its
          process group was killed." (or: stopped by the operator's kill
          switch / because the server is shutting down) + output so far

never: signal from another thread · signal after reaping · wait forever
kept:  a job backgrounded on purpose outlives the call AND a restart
A restart, measured and fixed (2026-10-08)·text
before  kickstart -k -> SIGTERM launcher
        no ExitTimeOut -> launchd SIGKILL at 5 s, children orphaned
        launcher's own 3 s cut -> turns killed ~1 s into a 30 s grace
        -> no pipeline's finally ever ran

after   shutdown deadline = turn grace + uvicorn window,
          never past ExitTimeOut - 2 s (warned at startup if too short)
        office plist ExitTimeOut 45
        turns still queued for the Chief -> withdrawn as "interrupted"
        untaken turns come back:
          Telegram  tg-<chat>-<message>  ask status, resend same id if missing
          WebUI     webui-<uuid>         words return to the queue, held
        waiting turn reads pending, not 404 (a 404 invited a double run)

Exercise

Restart a service you run while it is doing real work, under its real supervisor, and time each layer: how long the supervisor waits, how long your process waits, how long the work gets. Then list what was running and where each piece ended up: finished, cut silently, orphaned, or returned to its owner.
Hint
The supervisor's timer usually wins. If you have never set it, the grace in your code is a number nobody ever got.

Progress

Progress is local-only — sign in to sync across devices.
Spotted a bug or have feedback on this page?Report an Issue

Comments 0

🔔 Reply notifications (sign in)
Sign in — Please sign in to comment.

No comments yet — be the first.