Skip to content
C.W.K.
Lesson 01 of 04 · published

One Process Owns Everything

~13 min · launchd, keepalive, uvicorn, fastapi-lifespan, startup-order

Level 0Mise en Place
0 XP0/43 lessons0/12 achievements
0/100 XP to next level100 XP to go0% complete

One label, one job

Sizzle in production is one launchd agent, com.cwk.sizzle-serve, on the registered service host. Its plist is short, and every key in it is a decision. RunAtLoad starts the job when the agent is loaded. KeepAlive starts it again whenever it exits, for any reason. A ThrottleInterval of 10 keeps a crash loop from respawning more often than every ten seconds. ProgramArguments runs the cwk-sizzle conda environment's own Python with -m backend.serve, and WorkingDirectory is the repository root, so the import path is the repo and no shell profile is involved. Standard output and standard error go to sizzle-db/logs/ under the family data root.

Look at the values and you will see @@HOME@@ and @@DATA_ROOT@@ where paths should be. launchd does no shell expansion of a plist's values: a $HOME written into a path stays those five characters, and the plist's own EnvironmentVariables block only sets what the started process sees. Every path has to be written out in full, so a plist kept in a repository and shared by every Mac can only be a template. The installer renders it for the Mac it runs on, which is the next lesson. For now the point is that the tracked file names no machine and no user.

What the process is

backend/serve.py is the whole entry point. It hands uvicorn the import string backend.app:app with a host and a port from backend/settings.py. The host defaults to 127.0.0.1. The port defaults to engine_port("cwkSizzle"), which reads the family registry's port ledger and answers 7600. Both can be overridden (SIZZLE_HOST, SIZZLE_PORT), which keeps a second instance off production's port. They do not keep it off production's data. The data directory is its own override, SIZZLE_DATA_DIR, and without it a second instance on another port opens the same journal as the first. The test suite relies on none of these variables: it hands create_app a temporary directory (lesson 3). There is no workers argument, so this is one process with one event loop.

The registry row that matters, with its host written as a role, is {'host': <role>, 'engine': 7600, 'https': 7601}. The engine only ever listens on loopback 7600. Port 7601 is the Tailscale Serve HTTPS twin in front of it, and that is how the phone and other Macs reach Sizzle. The engine keeps no copy of the registry of its own: backend/family_registry.py loads the vendored scripts/kit_family_endpoints.py by file path, the same file the installer runs.

Everything that lives inside it

PartWhere it livesWhat it does
HTTP APIroutes in backend/app.pyThe routes the web client and the iPhone app both call
Web clientmount_webui(app, frontend/dist), mounted lastServes the built React app; API routes win because they are registered first
Reading workerReadingWorker, an asyncio taskReads pending plates, two at a time by default (SIZZLE_READING_CONCURRENCY)
Conversation turnsThreadsSends Dad's questions to cwkPippa and records the answers
Staged uploadsstaged/ in the data directoryHolds each photo uploaded on its own before the plate that names it
HealthGET /api/healthAnswers ok, app: "sizzle" and the product version

The health route is the one Firelink's declaration for Sizzle probes, and the expected body is the literal "app":"sizzle". A different engine answering 200 on the same port still fails that probe. firelink-quest's one-owner-per-fact track teaches why the family registry, and not the app, owns declarations like this one.

Startup order is a contract

FastAPI's lifespan is the only startup code, and its order is deliberate.

  1. Rebuild the mirror. sizzle.open replays journal.jsonl into the SQLite mirror on a worker thread (asyncio.to_thread) and logs how many events it read. Nothing is served before the mirror is whole.
  2. Declare the brain slots. brain_control.adopt_once registers Sizzle's per-function slots with the family authority before any job resolves one, so a startup re-read cannot race its own registration. An unreachable authority only logs a warning; resolving a slot later declares it lazily.
  3. Wire the worker. on_reading_requested becomes worker.kick, so every capture, correction or re-read request wakes it.
  4. Start the worker. It first records a reading.requested for every plate whose current reading predates the rubric in force, then starts its loop and kicks itself once. That drains whatever was pending when the last process stopped.
  5. Resume turns. threads.resume looks at every unfinished conversation turn. If cwkPippa already holds the answer for that turn's request id, the answer is recorded. Otherwise the turn is marked failed with Interrupted by an engine restart. Ask again. It never resends a question.

Shutdown runs the other way: stop the worker, cancel the turn tasks, close the store.

Why KeepAlive is enough

Sizzle runs no scheduled jobs. The reading worker wakes once at every start, on captures, corrections and re-read requests, when the reading slot gets a new brain, and after a reading that left its plate pending. It never wakes on a clock. So the engine needs exactly one guarantee from outside: if it dies, start it again. KeepAlive is that guarantee, and the lifespan does the rest. The journal is the record and the mirror is disposable. A plate stays pending in the mirror until a reading is recorded or fails, so a reading cut off by a crash is simply read again by the next process. Unfinished turns are settled honestly. A crash costs the model call that was in flight and nothing that was recorded.

Principle: Make the recovery path the start path. When every start rebuilds derived state from the record and picks up queued work, a restart is an ordinary boot, and a supervisor that only restarts is all the supervision you need.

swift-quest's macos-citizen track covers launch agents in general. The lesson here is the pairing: a supervisor that does one thing, and a process whose every start is a recovery.

Code

cwkSizzle, deploy/com.cwk.sizzle-serve.plist: the launchd job, a template with @@HOME@@ and @@DATA_ROOT@@ tokens (verbatim, lines 4-32)·text
<dict>
	<key>EnvironmentVariables</key>
	<dict>
		<key>HOME</key>
		<string>@@HOME@@</string>
		<key>PATH</key>
		<string>@@HOME@@/miniconda3/envs/cwk-sizzle/bin:/opt/homebrew/bin:/usr/local/bin:/usr/bin:/bin</string>
	</dict>
	<key>KeepAlive</key>
	<true/>
	<key>Label</key>
	<string>com.cwk.sizzle-serve</string>
	<key>ProgramArguments</key>
	<array>
		<string>@@HOME@@/miniconda3/envs/cwk-sizzle/bin/python</string>
		<string>-m</string>
		<string>backend.serve</string>
	</array>
	<key>RunAtLoad</key>
	<true/>
	<key>StandardErrorPath</key>
	<string>@@DATA_ROOT@@/sizzle-db/logs/serve-stderr.log</string>
	<key>StandardOutPath</key>
	<string>@@DATA_ROOT@@/sizzle-db/logs/serve-stdout.log</string>
	<key>ThrottleInterval</key>
	<integer>10</integer>
	<key>WorkingDirectory</key>
	<string>@@HOME@@/CWK/cwkPersonalProjects/cwkSizzle</string>
</dict>
cwkSizzle, backend/serve.py: the whole entry point (verbatim, lines 1-15)·python
"""Run the Sizzle engine: ``python -m backend.serve`` from the repo root."""

from __future__ import annotations

import uvicorn

from . import settings


def main() -> None:
    uvicorn.run("backend.app:app", host=settings.HOST, port=settings.PORT, log_level="info")


if __name__ == "__main__":
    main()
cwkSizzle, backend/app.py: the lifespan, in startup order (verbatim, lines 119-138)·python
    @asynccontextmanager
    async def lifespan(_app: FastAPI):
        count = await asyncio.to_thread(sizzle.open)
        logger.info("sizzle: mirror rebuilt from %d journal events", count)
        # Declare the per-function brain slots before any job resolves one,
        # so a startup re-read never races its own registration. An
        # unreachable authority only delays it: resolve() declares lazily.
        try:
            await asyncio.to_thread(brain_control.adopt_once)
        except Exception as exc:  # noqa: BLE001
            logger.warning("sizzle: brain slots not declared yet: %s", exc)
        sizzle.on_reading_requested = worker.kick
        worker.start()
        await threads.resume()
        try:
            yield
        finally:
            await worker.stop()
            await threads.close()
            sizzle.close()

External links

Exercise

Take a small web service you run on your own Mac and give it this shape. Write a LaunchAgent plist with RunAtLoad, KeepAlive, a ThrottleInterval and log paths in a directory you create first. In the service, move every piece of derived state (a cache, an index, a search table) into a startup step that rebuilds it from your source of truth and logs how much it read. Then kill the process with kill -9 and confirm that launchd started a new one and that the log shows a fresh rebuild.
Hint
launchctl print gui/$(id -u)/your.label shows the job's state, its last exit status and its PID; compare the PID before and after the kill. If the job never starts at all, check that the log directory exists: launchd does not create it.

Progress

Progress is local-only — sign in to sync across devices.
Spotted a bug or have feedback on this page?Report an Issue

Comments 0

🔔 Reply notifications (sign in)
Sign in — Please sign in to comment.

No comments yet — be the first.