The Boundary That Was Erased

"Zero-copy and no isolation are not two features. They are one erasure, seen from two sides."

What the Boundary Used to Do

On a discrete-GPU machine the card's memory is a fence. A process that fills VRAM cannot take the host's memory; a process that hangs the GPU cannot stop the CPU from noticing and, if it must, resetting the device. The copy across PCIe that the previous lessons priced as a cost was also a quarantine: whatever happened on the far side of the bus stayed there. Unified memory removed that bus, and with it the quarantine. One pool means one arbitration layer deciding who gets the GPU and who gets the memory, and there is no hardware line that stops one claimant from taking everything — the display included. This is the operating cost the founder's case against the design named in track two, and this lesson pays it in full, because the household has been paying it for thirteen years.

A Thirteen-Year Inheritance

The household's first sustained GPU stall was on a 2013 Mac Pro with two AMD FirePro cards: under sustained compute with several claimants, the driver would lose a card — a "GPU not found" panic — and the machine would restart. The same failure, with different symptoms, followed through an iMac Pro with a Vega GPU, a modular Mac Pro with Radeon Pro cards, and every Ultra Mac Studio since: two GPU vendors, discrete to on-die, split memory to unified. On Apple silicon it no longer panics. An on-die GPU cannot be "not found", so the same arbitration failure arrives as a hang — one claimant holds the Metal queue, the rest starve, the kernel stays up and the screen does not. Same parent class, different child.

What survives four hardware generations is not in the hardware. The common factor is macOS's GPU arbitration under a specific workload: sustained compute, sleep and wake cycles, several claimants at once. That workload was rare until frameworks like MLX made hours-long GPU compute ordinary on a Mac — and the population that runs it is a few thousand people at most, which is why the signal sits below the threshold at which a platform vendor acts. "Fixed", over those years, has always meant "below the pain threshold": once-a-day became months apart, and the adapted users stopped reporting. Every major macOS release re-lays the path, so the risk is redrawn annually.

Not a Lottery, and Not the Dies

Two wrong diagnoses are worth retiring by name. The first is "silicon lottery" — the idea that some units are defective. The household's sample is eight Ultra-class machines over a decade, all reproducing the behaviour; under a lottery with even a 5% defect rate, eight failures is a probability with ten zeros after the decimal point. Eight machines are eight reproductions, not eight coincidences. Apple's own conduct agrees: no recall, no repair programme, the same fused design carried into the next generation. The second wrong diagnosis was the author's own, and it belongs in a confession callout below: describing the Ultra stall as "a bug between the two dies". It is not. The dies are joined by a fabric Apple says makes them "behave as a single unified processor"; what fails is the software arbitration layer over that topology, which has the fewest walkers of any path in macOS. Naming the hardware was wrong, and the correction matters, because a hardware defect could be replaced and a software side effect of the design cannot — it is the same erasure that makes 512 GB local inference possible.

Living With It: The Claimant's Discipline

Since nothing in the hardware fences a GPU claimant, the fence has to be the claimant. Two knobs matter. The operating system publishes a recommended working set for the GPU — "an approximation of how much memory … this GPU device can allocate without affecting its runtime performance" — 464 GiB on office (90.6% of 512), 17.8 GiB on air (74% of 24). And MLX lets a process cap itself: set_memory_limit, set_cache_limit, set_wired_limit. The measured surprise is where MLX's default limit lands: the allocator sets it to the smaller of 1.5 × the recommended working set and 95% of physical memory, and on both lab Macs the 95% cap wins — 486.4 GiB on office, 22.8 on air — which is above the operating system's recommendation on both machines. A well-behaved inference server sets its limit to the recommended figure or below, leaves cache limits sane, and never sleeps the machine mid-run. The household's operating rules, learned the expensive way: keep the machine awake, reboot on symptom rather than debug, do not chain Ultras into one job (a population of zero never gets a fix), and skip the first build of each major macOS. Loss radius: one reboot. An early-adopter tax, and cheap at that.

Code

gpu_citizen.py — a claimant that fences itself, since nothing else will·python

#!/usr/bin/env python3
"""Read the OS's recommended GPU working set and MLX's default memory limit,
then cap this process at the recommendation. Run on any Apple silicon Mac."""
import mlx.core as mx

info = mx.device_info()
phys = info["memory_size"]
rec = info["max_recommended_working_set_size"]

default_limit = mx.set_memory_limit(rec)      # returns the previous (default) limit
print(f"{info['device_name']}: physical {phys/2**30:.0f} GiB")
print(f"  recommended working set : {rec/2**30:6.1f} GiB  ({rec/phys:.1%} of physical)  <- the OS's advice")
print(f"  MLX default memory limit: {default_limit/2**30:6.1f} GiB  ({default_limit/phys:.1%})  <- above the advice")
print(f"  this process now capped : {rec/2**30:6.1f} GiB")

mx.set_cache_limit(4 * 2**30)                 # keep freed buffers, but not without bound
x = mx.zeros((1 << 28,), dtype=mx.float32); mx.eval(x)
print(f"  active {mx.get_active_memory()/2**30:.2f} GiB  peak {mx.get_peak_memory()/2**30:.2f} GiB  cache {mx.get_cache_memory()/2**30:.2f} GiB")

# office, M3 Ultra, 2026-09-15: physical 512, recommended 464.0 (90.6%), MLX default 486.4 (95%)
# air,    M3,       2026-09-15: physical  24, recommended  17.8 (74%),   MLX default  22.8 (95%)

The kernel-side knob, and the household's operating rules·bash

sysctl iogpu.wired_limit_mb
# iogpu.wired_limit_mb: 0        <- default on every fleet Mac: the OS chooses the GPU's wired ceiling
# Raising it (sudo sysctl iogpu.wired_limit_mb=<MB>) lets the GPU wire more of the pool;
# it is documented only in the mlx-lm README, and it moves the fence, it does not add one.

# Operating rules for a Mac that serves GPU inference all day (learned, not read):
#   pmset -a sleep 0 disablesleep 1     # never sleep the machine mid-run
#   reboot on symptom; do not debug a hung Metal queue
#   one job per Ultra; do not chain Ultras into a single job
#   skip the .0 build of each major macOS on the inference host

Exercise

Run gpu_citizen.py on your Mac and add three numbers to your card: physical memory, the recommended working set, and MLX's default limit. Then answer: if you ran an inference server at MLX's default on your machine and a second GPU process started, which process's discipline decides whether the display keeps drawing — and what would a discrete-GPU machine have done instead?

Hint

Neither process has a fence; the one that allocates last discovers the pool is full, and the window server is a GPU process too. A discrete-GPU machine would have refused the allocation at the VRAM boundary and left the host untouched. That refusal is the boundary this lesson is about.