"The server stopped being the only machine that runs a job. It never stopped being the only machine that decides what a job is."
The Week the Engine Went Wide
Until 2026-09-21 the engine ran its inference on one server — by design from the start, and by hard rule since mid-August. On that date two Windows machines with NVIDIA GPUs joined as workers, running the engine's worker under WSL. The next day five Apple Silicon Macs joined too. Eight machines now run jobs: the server, the two CUDA workers, and the five Macs. The development machine is still not one of them; it stays code, docs, and tests. Going from one host to eight could have scattered the engine's identity across eight disks. It did not, because of a split this track has been teaching since its fourth lesson.
Authority Stays Home
The server kept everything that defines a job: every model ID, the registry, the public API, the queue, and every archive. A worker owns none of that. It owns a device and a verified copy of some files, and it exposes nothing but inference. It never mints an ID, never keeps a second catalog, never archives a result — finished images and videos travel back to the server, which checks them and files them. Clients never learn a worker's address either. They talk to the server and name a machine, or ask the server to choose one. Execution spread out. Authority did not move an inch.
The Mirror Rule, Made Mechanical
The ground-truth lesson taught a habit: a mirror's emptiness is not evidence of absence, so check the source. With seven copies in play, five of them partial, a habit was not enough, and the rule became machinery in both directions. Absence is declared: each Mac worker advertises the verified subset of models it holds, and placement, direct dispatch, and submission all refuse a model outside that inventory. Presence is not trusted either: before a job runs anywhere but the server, every file it needs is hashed on the server and hashed again on the worker, and a mismatch refuses the job. Copies flow one way — a sync that runs only from the server, never deletes anything on the far side, and will not copy under a running worker. It is the tensors-don't-lie rule from the adapter track, applied across machines: a file's identity is its content, checked at both ends.
One Gate per Device, One Line per Machine
The device-gate lesson hoisted the GPU permit to process scope. With many machines, that turned out to be exactly the right unit: every worker is its own process with its own gate, so there is one permit per device without anyone arranging it. A new layer went on top instead of replacing anything. The server keeps a first-in-first-out line per machine, and a job takes its place in that line before its files are hashed or its model is loaded — so a small job cannot slip past a larger one queued earlier on the same machine just because its preparation finished first.
Placement Is the Decision That Earned a Brain
Every job now answers one more question: which machine? By default the answer is the server. A job can name another machine explicitly, and then it runs there or nowhere — no silent failover, ever: an unsupported choice is refused, and an unreachable machine is an error, not a reason to try a different one. Or a job can ask for Auto, which is opt-in. Auto filters first: the machine must be online, hold the model, be compatible with what it is keeping loaded, support the model's family, and have the memory for the job. Then it ranks the survivors by earliest estimated finish — the work already queued there, plus this job's estimated run time including a cold model load, plus the cost of reloading any warm model this job would push out. It learns as it goes, replacing its starting guesses with measured times per machine and model, and it makes the choice and the queue entry in one atomic step, so two Auto jobs submitted together see each other.
Residency Has Two Physics
Keeping a model loaded is worth real time: a cold load of a large instruction-editing model takes minutes. So residency became policy, and the policy follows the hardware. Machines with unified memory — the server and the Macs — keep a protected set of warm models, admitted by memory headroom rather than by a fixed count. The CUDA workers keep exactly one checkpoint and clear their caches before loading a replacement. A machine's residency cannot change while it has work queued or running, and any reservation on a machine keeps video off it, because video's runtime needs the memory the image models are holding.
War Story: A Guard on the Wrong Side of the Fence
The first cross-backend bug ran in the direction nobody warns about. A memory guard written for unified memory — cap the allocator at a fraction of the shared pool so the operating system keeps its breathing room — ran inside a shared upscale path on every backend. On a CUDA worker it set a process-wide allocator limit that outlived the upscale. The next large model load on that 24 GB card failed with gigabytes still free, and the failure looked exactly like a model too big for the card. The fix kept the guard on the backend it describes; a dedicated GPU's memory is managed by the card's real capacity, one job at a time, and explicit eviction.
Nothing Runs Twice
A server restart can land in the middle of a job. The engine now journals every job, and after a restart anything unfinished is marked interrupted — never replayed. A job that may already have run must not run again behind the artist's back; on the workspace side the same rule keeps paid requests from ever being retried. And when a worker runs out of memory, the failure is recorded against that kind of workload, so an identical repeat can be refused instead of crashing the same way twice.