A Phone GPU's Habits

"Apple's GPU was never a graphics card that lost its slot. It was a phone GPU that gained a memory bus."

Tile-Based, Because a Phone Cannot Afford Anything Else

Desktop GPUs from the discrete-card tradition render a frame by streaming every triangle through the pipeline and writing pixels to a framebuffer in memory, over and over, relying on a wide, hot memory bus to absorb the traffic. A phone GPU cannot: memory traffic is the most expensive thing a battery-powered chip does. So Apple's GPUs, like the mobile GPUs they descend from, use tile-based deferred rendering — Apple's own documentation title says it plainly: "Tailor your apps for Apple GPUs and tile-based deferred rendering". The screen is cut into tiles, each tile is rendered entirely in a small on-chip memory, and only the finished tile is written out. Bandwidth to main memory falls by a large factor; the price is a different set of habits for programmers, and a GPU whose instincts are about not touching memory.

Those instincts came to the Mac intact. The same on-chip tile memory is exposed to compute work as threadgroup memory; the same discipline — keep the working set close, touch DRAM once — is what a well-written Metal kernel or an MLX matrix multiply does. And the same lineage means Apple's GPU has no separate memory of its own to manage. It never did. On a phone the GPU always read the same DRAM as the CPU, and unified memory on the Mac is that arrangement scaled up — on the Ultra, sixteen memory channels' worth of bus instead of a phone's two, by the bandwidth arithmetic (Apple publishes bandwidth, not channel counts; the Air's 100 GB/s is the two-channel fit).

What the Pool Actually Delivers to the GPU

Spec bandwidth (track one) is what the memory can supply. What a GPU kernel achieves is a different number, and it is the number that bounds decode. The code block measures it STREAM-style on each lab Mac: a large float32 array, an elementwise add (read and write) and a sum (read only), best of five.

Alias	Chip	Spec	Add (read+write)	Sum (read)	Achieved ÷ spec	Evidence
air	M3	100 GB/s	93 GB/s	97 GB/s	93–97%	measured 2026-09-15
pro2023	M3 Max	400	359	391	90–98%	measured
music	M2 Ultra	800	740	734	92%	measured
office	M3 Ultra	819	635	638	78%	measured

Three of the four reach the spec within a few per cent, which is remarkable for a single kernel and a fair vindication of the bus arithmetic. The M3 Ultra does not: it delivers about 78% of its spec, less in absolute terms than the M2 Ultra it replaced, on the same code and the same MLX build. The lab track's decode ladder shows the same thing from the other side — the M2 Ultra decodes a 27B model slightly faster than the M3 Ultra. This quest does not know why. The plausible places to look are the fabric between eighty GPU cores on two dies and sixteen memory controllers, and how a single stream of requests is spread across it; none of that is observable from user space, and Apple publishes nothing about it. So the quest reports the ratio, labels it, and uses the achieved figure, not the spec, when it predicts decode on office.

Cores, Then Bandwidth, Then Nothing Else

An Apple GPU is described by its core count — 10 on the M3, 40 on the Max, 80 on the Ultra — and each core holds execution units, registers and a slice of that on-chip memory. For rendering and for prefill, cores are what you are buying: more of them process more pixels or more prompt tokens per second, and the M3 Ultra's prefill lead over the M2 Ultra in the lab track is the newer, larger GPU showing. For decode, cores stop mattering once there are enough of them to keep the memory bus busy, and the table above is the bus. Everything else in a GPU spec sheet — clock, ray-tracing hardware, mesh shaders — is real and irrelevant to a language model.

Code

stream.py — achievable GPU memory bandwidth, STREAM-style, with MLX·python

#!/usr/bin/env python3
"""stream.py — achievable GPU memory bandwidth, STREAM-style, with MLX.
Two kernels on a large float32 array: an elementwise add (read + write) and a
sum reduction (read only). GB/s = bytes moved ÷ best-of-five seconds."""
import sys, time
import mlx.core as mx

gib = float(sys.argv[1]) if len(sys.argv) > 1 else 4.0
n = int(gib * 2**30 / 4)
a = mx.ones((n,), dtype=mx.float32); mx.eval(a)


def best(fn, runs=5):
    fn(); t_best = 1e9
    for _ in range(runs):
        t = time.perf_counter(); fn(); t_best = min(t_best, time.perf_counter() - t)
    return t_best


t_add = best(lambda: mx.eval(a + 1))          # read n·4 B, write n·4 B
t_sum = best(lambda: mx.eval(mx.sum(a)))      # read n·4 B
info = mx.device_info()
print(f"{info['device_name']:16} array {gib:.0f} GiB  add {2*n*4/t_add/1e9:5.0f} GB/s   sum {n*4/t_sum/1e9:5.0f} GB/s   mlx {mx.__version__}")

# 2026-09-15, mlx 0.32.2, macOS 26.6.2:
# Apple M3         array 2 GiB  add    93 GB/s   sum    97 GB/s   (air, spec 100)
# Apple M3 Max     array 4 GiB  add   359 GB/s   sum   391 GB/s   (pro2023, spec 400)
# Apple M2 Ultra   array 4 GiB  add   740 GB/s   sum   734 GB/s   (music, spec 800)
# Apple M3 Ultra   array 4 GiB  add   635 GB/s   sum   638 GB/s   (office, spec 819)

What macOS says about the GPU it is driving·bash

system_profiler SPDisplaysDataType | grep -E "Chipset Model|Total Number of Cores|Metal"
# Chipset Model: Apple M3 Ultra
# Total Number of Cores: 80
# Metal Support: Metal 4

python -c "import mlx.core as mx; print(mx.device_info())"
# {'device_name': 'Apple M3 Ultra', 'max_recommended_working_set_size': 498216206336,
#  'memory_size': 549755813888, 'architecture': 'applegpu_g15d', ...}

Exercise

Run stream.py on your Mac (pick an array size well under a quarter of your memory) three times and add the best add and sum figures to your card beside the spec. Compute achieved ÷ spec. Then, with the physics track's decode-ceiling formula (bandwidth ÷ bytes per token), predict the decode ceiling for a model that reads 4.47 GB per token (the lab's 9B at 4 bits) using the spec and again using your achieved figure, and keep both — the lab track will tell you which one was closer.

Hint

If your ratio is above 90%, your Mac is in the majority. If it is nearer 78%, you probably have an M3 Ultra, and you have just reproduced a finding this quest could not explain — write down the mlx version, because a future release could change it.