Why Apple Silicon Unified Memory Changes Everything

One pool of RAM, addressed by two processors

The single most important architectural difference between an NVIDIA discrete GPU and Apple Silicon isn't peak FLOPs or vendor allegiance. It's where the data lives.

Where the numbers come from: NVIDIA lists the 4090's 384-bit memory bus but not its bandwidth, so 1,008 GB/s is derived (21 Gbps GDDR6X × 384 bits ÷ 8). Apple publishes 819 GB/s but doesn't name the memory standard, and that one figure can't name it either: 16 channels of LPDDR5-6400 and 12 of LPDDR5X-8533 both come to 819.2 GB/s. The rest of the M3 family settles it, assuming the family shares one memory speed. At 6400 MT/s a 64-bit channel carries 51.2 GB/s, so Apple's figures for the M3 (100), the M3 Pro (150) and the two M3 Max configurations (300 and 400) are 2, 3, 6 and 8 channels (102.4, 153.6, 307.2 and 409.6 GB/s) written rounded down. No LPDDR5X speed lands all four on whole channel counts. Hence LPDDR5.

On the desktop, you allocate with cudaMalloc on the GPU and copy data over PCIe. The CPU and GPU each have their own DRAM, joined by a ~32 GB/s link that is about thirty times narrower than the GDDR6X the GPU reads from. The discipline on a discrete card follows from that: get the data across once, then keep it on the card.

On Apple Silicon, the System-on-Chip integrates CPU cores, GPU cores, and DRAM controllers behind one memory subsystem. device.makeBuffer(length:options:) with storageModeShared returns a buffer that the CPU can read and write directly, that the GPU sees at the same physical address. No copy. No PCIe.

The first consequence is mechanical: cudaMemcpy disappears. The 9-step GPU workflow you'll meet in Track 5 collapses to 7 on Apple Silicon — steps 4 and 7 are no-ops.

Where unified memory wins, and where it doesn't

"No PCIe" invites a tempting conclusion: Apple must be faster. For the case most people run, that's backwards. When a model fits in VRAM, its weights cross PCIe once, at load, and stay on the card. Every token after that is generated by reading those weights out of GDDR6X — the link never enters the loop. Decode speed is set by memory bandwidth, and the 4090 has about 23% more of it.

• The model fits on both: the 4090 decodes faster. An 8B model in FP16 is ~16 GB. Bandwidth divided by the bytes read per token puts the ceiling near 63 tokens/s on the 4090 and 51 on the M3 Ultra. Real numbers land below both ceilings — compute and the KV cache take their share — but the order doesn't flip.
• The model doesn't fit in 24 GB: unified memory wins, and it isn't close. Llama 70B in FP16 is ~141 GB. The desktop in the table holds 88 GB in total (24 GB VRAM plus 64 GB DDR5), so it can't hold those weights at all without paging from disk. A 192 GB Mac Studio recommends up to about 161 GiB for its GPU (measured on an M2 Ultra, macOS 26.6.2), so all ~131 GiB of weights sit where the GPU reads them. That's a capacity win, not a speed win: at the M2 Ultra's 800 GB/s the ceiling is 800 ÷ 141 ≈ 5–6 tokens/s.
• Offload is where PCIe really bites. Give the desktop enough RAM and the layers that don't fit in VRAM still have to go somewhere: they either run on the CPU against dual-channel DDR5 (~90 GB/s at DDR5-5600) or get streamed to the GPU over the ~32 GB/s link. Either way the overflow reads memory an order of magnitude slower than the GPU's own. Unified memory never takes that detour, and it makes no second copy into a separate VRAM pool at load either.
• The link also shows up when a model is split across two cards — the 4090 has no NVLink, so they talk over PCIe, which is cheap for a pipeline split that hands one activation across each boundary and expensive for tensor parallelism, which synchronizes inside every layer — and in host code that pulls tensors back to the CPU every step.
• Apple's side has its own trade-off: that 819 GB/s is shared. If the CPU and GPU both hammer memory at once, each gets less of it. On a discrete GPU, CPU traffic runs on a different bus.

This is also why MLX (Apple's array framework for machine learning) never asks which device an array lives on: every array already sits in the one pool, and you pick the device per operation instead of moving arrays around. Don't stretch "unified" into "nothing ever copies", though. mx.array(np_array) copies the NumPy buffer into memory MLX allocates itself — MLX's documentation says so, and changing the NumPy array afterwards leaves the MLX array as it was. The zero-copy path runs the other way: np.array(mx_array, copy=False) gives NumPy a view of MLX's memory. Unified memory removes the hop between processors; it doesn't remove the copies a library decides to make.