Why Apple Silicon Unified Memory Changes Everything

One pool of RAM, addressed by two processors

The single most important architectural difference between an NVIDIA discrete GPU and Apple Silicon isn't peak FLOPs or vendor allegiance. It's where the data lives.

System	CPU memory	GPU memory	Transfer
RTX 4090 desktop	64 GB DDR5	24 GB GDDR6X	PCIe 4.0 ×16, ~32 GB/s
Mac Studio M3 Ultra (512 GB)	512 GB LPDDR5X (shared)		Zero — same RAM

On the desktop, you allocate cudaMalloc on the GPU and copy data over PCIe. The CPU and GPU each have their own DRAM; they meet through a 32 GB/s pipe that often becomes the bottleneck for inference.

On Apple Silicon, the System-on-Chip integrates CPU cores, GPU cores, and DRAM controllers behind one memory subsystem. device.makeBuffer(length:options:) with storageModeShared returns a buffer that the CPU can read and write directly, that the GPU sees at the same physical address. No copy. No PCIe.

Practical consequences:

Models that don't fit in 24 GB VRAM often fit in 192–512 GB unified memory. Llama 70B in FP16 (~140 GB) is impossible on a 4090 without offloading; trivial on a 192 GB Mac Studio.
cudaMemcpy disappears. The 9-step GPU workflow (we'll see in Track 5) collapses to 7 on Apple Silicon — steps 4 and 7 are no-ops.
Bandwidth differs. 4090's GDDR6X = ~1.0 TB/s, M3 Ultra's LPDDR5X = ~820 GB/s. Apple is slightly slower per byte but skips PCIe entirely. Net throughput often wins.
One trade-off is real: that bandwidth is shared between CPU and GPU. If both pound it simultaneously you lose effective bandwidth. On a discrete GPU, CPU traffic is on a different pipe.

This is also why MLX (Apple's PyTorch-shaped framework) feels different to use. mlx.array(np_array) doesn't move bytes — it's a view onto the same RAM the numpy array already lives in.

Code

Allocate a buffer the CPU and GPU both see·swift

import Metal

let device = MTLCreateSystemDefaultDevice()!
let n = 1024
let bytes = n * MemoryLayout<Float>.stride

// storageModeShared: same physical RAM, CPU + GPU both addressable.
let buf = device.makeBuffer(length: bytes, options: .storageModeShared)!

// CPU writes directly — no staging, no copy.
let ptr = buf.contents().bindMemory(to: Float.self, capacity: n)
for i in 0..<n { ptr[i] = Float(i) }

// GPU launches a kernel against the same buffer.
// (encoder.setBuffer(buf, offset: 0, index: 0) ... )
// On a 4090 the equivalent is malloc + cudaMalloc + cudaMemcpy.

Storage modes — the three Metal options·swift

// shared    — CPU + GPU same RAM, no copy, default for compute.
.storageModeShared

// private   — GPU-only memory (still in unified RAM, but not
//             CPU-visible). Faster GPU access on some shapes.
.storageModePrivate

// managed   — Intel Mac with discrete GPU only. Apple Silicon
//             ignores this; behaves like shared.
.storageModeManaged

Exercise

On any Mac you can SSH into, run system_profiler SPHardwareDataType | grep Memory (gives unified RAM size) and system_profiler SPDisplaysDataType | grep VRAM (will say 'Built-In' on Apple Silicon, listing the same number). Now compare to nvidia-smi --query-gpu=memory.total --format=csv output you saved in lesson 2 of the previous track. Internalize the gap: 24 GB siloed VRAM vs 192-512 GB unified — same workload, different physics.

Why Apple Silicon Unified Memory Changes Everything

One pool of RAM, addressed by two processors

Code

External links

Exercise

Progress

Comments 0