~14 min · lab, m2-ultra, m3-ultra, gpu-generation, stream, measured
Level 0Spec-Sheet Skimmer
0 XP0/91 lessons0/19 achievements
0/100 XP to next level100 XP to go0% complete
"Two Studios, two years apart, the same 800-odd gigabytes a second. The newer one wins the stage that is compute and loses the stage that is memory — on every model, by the same margin."
The Second Controlled Experiment
The M3 ladder held the microarchitecture fixed and varied the count. This lesson holds the memory nearly fixed and varies the generation: the music Studio's M2 Ultra (76 GPU cores, 800 GB/s, 192 GB) against the office Studio's M3 Ultra (80 cores, 819 GB/s, 512 GB). Apple's own claim for the newer chip is that its GPU brings "up to 2x faster performance than M2 Ultra" with a "next-generation GPU" architecture — Dynamic Caching, hardware ray tracing, mesh shading — and that UltraFusion joins two M3 Max dies "as a single, unified chip". The bandwidth column, meanwhile, moved by two per cent. So the prediction is sharp: prefill, the compute stage, should improve; decode, the bandwidth stage, should not move. Same lab, same day, same models, same environment.
Model (4-bit)
music, M2 Ultra · prefill / decode
office, M3 Ultra · prefill / decode
Prefill ratio
Decode ratio
Evidence
Qwen3.5-0.8B
5,396 / 369.4
6,312 / 338.2
×1.17
×0.92
measured 2026-09-15
Qwen3.5-2B
3,055 / 275.8
3,992 / 257.1
×1.31
×0.93
measured
Qwen3.5-4B
1,333 / 153.3
1,748 / 147.6
×1.31
×0.96
measured
Qwen3.5-9B
789 / 105.5
1,043 / 95.1
×1.32
×0.90
measured
Qwen3.5-27B
233 / 35.3
315 / 32.6
×1.36
×0.92
measured
Qwen3.5-35B-A3B
986 / 101.5
1,276 / 89.1
×1.29
×0.88
measured
Half the Prediction Held
Prefill improved on every model, by 17 to 36% — a real generational gain in the compute stage, though nowhere near the 2× of the vendor's rendering claim, which is a claim about graphics. Decode did not merely fail to improve; it went backwards, by 4 to 12% on every model, most on the mixture. The older Studio decodes the 27B at 35.3 tokens per second to the newer one's 32.6. The bandwidth lesson's fits say the same in two numbers: the M2 Ultra pulls 546 GB/s of its 800 from a decode kernel, the M3 Ultra 497 of its 819. The stream test, which involves no model at all, says it a third way: 92% of spec on the M2 Ultra, 78% on the M3 Ultra. Three instruments, one ordering.
An Independent Witness
The lab is one runtime on one day, so it looked for a second. The llama.cpp community's long-running Apple-silicon table — different runtime, different model family, different year — lists the 76-core M2 Ultra decoding a 7B in F16 at 41.0 tokens per second and the 80-core M3 Ultra at 39.8, while prompt processing goes the other way, 1,402 to 1,538. Same pair, same direction — with smaller margins (about 3% on decode and 10% on prefill at F16, against this lab's 4–12% and 17–36% at 4 bits) — from people who never ran this lab. Why the newer memory system delivers a smaller share of its spec to a streaming read is not determinable from user space; Apple documents the die-to-die link's connections and bandwidth, not its behaviour under a memory-bound kernel. The quest reports the ratios, names the three instruments that agree, and declines to invent a mechanism. For the card: if you own the older Ultra and run models that fit it, decode is not a reason to upgrade; capacity and prefill are.
Code
lab_ultras.py — the two Studios side by side, model by model·python
#!/usr/bin/env python3
"""Same memory, newer GPU: the M2 Ultra (music, 800 GB/s, 76 GPU cores) beside the
M3 Ultra (office, 819 GB/s, 80 cores), model by model. Prefill is the compute
stage; decode is the bandwidth stage. Standard library only.
Usage: lab_ultras.py ladder-music.jsonl ladder-office.jsonl"""
import json, sys
def load(path):
rows = {}
for line in open(path):
r = json.loads(line)
if r.get("experiment") == "ladder":
rows[r["model"].split("/")[-1]] = r
return rows
a, b = load(sys.argv[1]), load(sys.argv[2])
ra, rb = next(iter(a.values())), next(iter(b.values()))
print(f"{'model':24} {'GB/tok':>6} | {ra['device']:>14} prefill decode | {rb['device']:>14} prefill decode | prefill decode")
for name in ["Qwen3.5-0.8B-4bit", "Qwen3.5-2B-4bit", "Qwen3.5-4B-4bit", "Qwen3.5-9B-4bit", "Qwen3.5-27B-4bit", "Qwen3.5-35B-A3B-4bit"]:
x, y = a[name], b[name]
print(f"{name:24} {x['weight_bytes_per_token']/1e9:6.2f} | {'':14} {x['prompt_tps']:7.0f} {x['generation_tps']:7.1f} | {'':14} {y['prompt_tps']:7.0f} {y['generation_tps']:7.1f} | "
f"x{y['prompt_tps']/x['prompt_tps']:4.2f} x{y['generation_tps']/x['generation_tps']:4.2f}")
print(f"\nspec bandwidth ratio {rb['spec_gbps']/ra['spec_gbps']:.2f} (M3 Ultra / M2 Ultra)")
# 2026-09-15, mlx 0.32.2 / mlx-lm 0.31.3:
# model GB/tok | M2 Ultra prefill decode | M3 Ultra prefill decode | prefill decode
# Qwen3.5-0.8B-4bit 0.42 | 5396 369.4 | 6312 338.2 | x1.17 x0.92
# Qwen3.5-2B-4bit 1.06 | 3055 275.8 | 3992 257.1 | x1.31 x0.93
# Qwen3.5-4B-4bit 2.37 | 1333 153.3 | 1748 147.6 | x1.31 x0.96
# Qwen3.5-9B-4bit 4.47 | 789 105.5 | 1043 95.1 | x1.32 x0.90
# Qwen3.5-27B-4bit 14.42 | 233 35.3 | 315 32.6 | x1.36 x0.92
# Qwen3.5-35B-A3B-4bit 1.66 | 986 101.5 | 1276 89.1 | x1.29 x0.88
# spec bandwidth ratio 1.02 (M3 Ultra / M2 Ultra)
If you can reach two Macs with similar bandwidth and different generations, run the ladder on both and build the side-by-side with lab_ultras.py. If you cannot, take your Mac's row from the community table linked above and compare its F16 decode against the ceiling for a 7B at 16 bits on your bandwidth. Write on the card which stage a newer generation would buy you, and by how much the table says.
Hint
Decode ratio near 1.0 between generations of similar bandwidth is the expected result; a prefill ratio well above 1.0 is the generational gain. If your decode ratio is far from 1.0, check that both machines were measured at the same context length and quantization — a mismatch there looks exactly like a generational difference.
Progress
Progress is local-only — sign in to sync across devices.