Control Plane vs Executor

"One host decides. One host does. They are not the same host on purpose."

Four Roles, Not Four Peers

Recall's work is spread across machines, but the split isn't 'more computers for more speed.' Each role is defined by what it's allowed to own:

The control plane — one always-on host. Owns the database, the job queue, accepted evidence, and every state transition. It is the single source of truth. When it says a job is done, the job is done.
The executor — another always-on host. Scans the archive, transcodes audio, and calls the paid transcription API. It owns no canonical state. Its local files are caches: rebuildable, disposable, never truth.
Storage (the NAS) — owns the original video bytes. Nobody copies those into a database.
The dev client (a laptop) — edits code, runs small tests, pushes to git. It never owns a job and never runs the heavy archive work.

Notice these are not interchangeable peers. They are roles with different rights, and the rights are the architecture.

Why Separate Deciding From Doing

The executor's job is everything slow, expensive, and failure-prone: a network filesystem that can hang, a multi-gigabyte video to transcode, a paid API call that might time out. That is exactly the kind of work you do not want fused to your source of truth. Fuse them and one stalled network mount can freeze your database; one crash mid-transcode can corrupt your queue.

Separate them, and the blast radius shrinks. The executor can crash, hang, or be killed mid-job — and the control plane's truth is untouched, because the executor was never allowed to write truth in the first place. It can only propose results, which the control plane accepts or rejects. Think brain and hands: the hands do the dangerous work, but the brain decides what actually happened.

The Executor Holds a Lease, Not the Truth

Concretely: the executor doesn't 'have' a job the way you own a file. It claims one from the control plane, works on it under a time-limited lease, and submits the result back for acceptance. If it dies, the lease expires and the control plane hands the job to someone else. Nothing is lost, because the executor never held anything that mattered. Track 6 is entirely about how that lease works; for now, hold the shape: heavy work lives on the host that owns nothing.

Code

Roles and rights (the split is the architecture)·text

CONTROL PLANE (one host)      EXECUTOR (one host)
  owns: database                owns: nothing canonical
  owns: job queue + leases       does: scan the archive
  owns: accepted evidence        does: transcode audio
  owns: every state change       does: call the paid API
  = single source of truth       caches: rebuildable, disposable

STORAGE (NAS)        owns original video bytes
DEV CLIENT (laptop)  edits code, pushes git, owns no job

The executor can crash mid-job — truth is untouched,
because it was never allowed to write truth.

Exercise

Look at any system you've built or used that does slow background work (a build server, a video export, a data import). Ask: does the thing that DOES the slow work also OWN the record of what's true? If yes, trace what happens when it crashes mid-work — does the truth survive? Sketch how you'd split it into a control plane that owns truth and a stateless executor that can crash safely.

Hint

The smell is a single process that both computes the expensive result AND writes the authoritative 'done' record. Pull those apart: let the worker compute and propose; let a separate owner accept. The test of a good split is 'kill the worker mid-job — is anything corrupted?' The answer should be no.