Scale-Out Is Not Scale-Up

"Another Mac adds a pool and a wall. The wall is 250 times narrower than the pool. That ratio is the whole difference between more machines and a bigger one."

Two Ways to Grow

A pool grows up — a wider bus, a denser die, the wire and density lessons — or out: another machine on a cable. The fleet's wires lesson priced the cable at 10 GB/s on Thunderbolt 5 against 2.5 TB/s across UltraFusion, and the big-models track read the public results: 1.3× at two Macs, 1.6× at four, on models that already fit one — and, in a separate five-Mac pipeline run on a different model, a fifth Mac buying nothing. The code block puts the two growths side by side. Scale-up raises the ceiling for every model that fits — a 1,229 GB/s M5 Ultra decodes the 27B at 85 where the M3 Ultra decodes it at 57, no cable involved. Scale-out raises the ceiling only for models that did not fit, and charges every layer of every token a crossing of the narrow wire. Two Studios are 996 GB of working set and a 250-to-1 wall between the halves; they are not a 1,024 GB Studio, and no cable Apple ships makes them one.

Apple's Own Scale-Out, and Its Population

Apple shipped RDMA over Thunderbolt for macOS 26.2 naming "distributed AI inference using MLX" as the use, and MLX's distributed backend followed within a week. The household's doctrine reads that move by its population rather than its press: the mini and the Studio are the two ends of Apple's desktop line, the mini buyers are the body of the market and the Studio buyers the tail, and when the mini is short of memory its buyers pair minis. So the multi-host marketing is aimed at mini pairs, where a chaining population large enough to surface and fix bugs can form; Ultra pairs are a tail within a tail. The signal that local AI has crossed from tail to body, in the doctrine's words, is mini pairs becoming common — not M5 Ultra reviews. That is a judgment, and the quest carries it as one; what the arithmetic adds is that a mini pair is two 64 GB pools behind the same 10 GB/s wall, which fits models that a single mini cannot and decodes them at the wire's price, exactly as the Studio pair would.

The Homework

Scale-out's homework is the wire. A wire ten times wider — 100 GB/s between Macs, a tenth of an NVLink — would put the per-layer crossing within reach of hiding behind a batch, and a coherent link like the rivals' C2C would put two Macs behind one page table, which is the two-tier lesson's shape with the second tier in the other box. Neither exists on a Mac, and the doctrine's reading is that neither will until the population that needs it is the body of the market. Until then, scale-out on a Mac is a loading path and a demonstration, and the household's fleet is built the other way: nine Macs that never share a token, each sized to the pool its job needs, joined by wires that carry requests and results — the fleet track — and never bytes per token. Scale up when the die arrives; scale out when the wire does; and until both, the pool is one package and the honest sentence is to say so.

Code

scale_out.py — N Macs: capacity adds, the wall adds, and the public speed column·python

#!/usr/bin/env python3
"""Scale-out is not scale-up. N Macs add capacity linearly and add per-token link crossings
linearly; the link is 250x narrower than the fabric. Public measurements for the speed
column (Geerling 2025-12-18, exo over RDMA); the capacity column is arithmetic."""
LINK, FABRIC = 10.0, 2500.0
for n, tok_s in [(1, 19.5), (2, 26.2), (4, 31.9)]:
    print(f"{n} M3 Ultra (Geerling's mix: 512 GB pairs and 256 GB pairs): capacity ~{n*498:5d} GB if all were 512 | measured 235B-A22B 8-bit decode {tok_s:4.1f} tok/s = x{tok_s/19.5:.2f} | per-token link crossings per layer: {n-1}")
print(f"\nlink / fabric = {LINK/FABRIC:.3f}: each added Mac adds a wall 250x narrower than the pool it adds.")
print("scale-up (a wider bus, a denser die) raises the ceiling for every model that fits; scale-out (another Mac) raises the ceiling only for models that did not fit, at the wire's price.")
print("the household's reading of the 'time' signal: not Ultra pairs but Mac mini pairs becoming common -- the body of the market pooling, where bugs get fixed.")

Exercise

Run scale_out.py with your own Mac's pool and the wire you actually have between two machines. For the model you most want, write whether it needs scale-up (a faster pool) or scale-out (a second pool) — and if the latter, the speed the wire would give it. Then write the doctrine's question on the card: is the population that would fix that wire's software you, or the body of the market?

Hint

If the model fits one Mac, scale-out cannot help and scale-up is a purchase decision for the next generation. If it does not, scale-out is possible at the wire's price and the household's verdict applies until the wire is ten times wider.