Skip to content
C.W.K.
Stream
Lesson 05 of 06 · published

NVIDIA Builds Unified Memory Too

~14 min · cuda, nvidia, unified-memory, gb10, gh200, convergence, vendor-claim

Level 0Spec-Sheet Skimmer
0 XP0/91 lessons0/19 achievements
0/100 XP to next level100 XP to go0% complete
"The company that built the wall now sells pools. Read the bandwidth-per-gigabyte column and the pools land beside the Mac, not above it — the shape was right, and the trade comes with the shape."

Three NVIDIA Pools

NVIDIA's own answer to the VRAM wall is the same answer Apple shipped in 2020: put a large memory behind a coherent link so the GPU can address it without a copy. Three products carry it. GH200 joins a Grace CPU with up to 480 GB of LPDDR5X to a Hopper GPU with HBM over NVLink-C2C at "900 gigabytes per second", and NVIDIA's page says the part that matters: "the CPU and GPU share a single per-process page table" — one address space, the GPU faulting pages in from the CPU's memory, exactly the journey track's mapped-touched-wired sequence without the copy step. GB200 NVL72 scales that to a rack. DGX Spark, the GB10 desktop, is the consumer-scale version: "128 GB LPDDR5x, coherent unified system memory" at "273 GB/s", the CPU and GPU on one board joined by NVLink-C2C with "5x the bandwidth of fifth-generation PCIe", 140 W. That is a Mac Studio's shape with NVIDIA's software on it, and the platform lesson explains why that combination sells: twenty years of CUDA code runs on it unchanged.

The Trade That Comes With the Shape

The code block's middle column is the one to read: bandwidth per gigabyte. A card is a small memory with an enormous bus — 56 GB/s per GB on a 5090. A pool is a large memory with a modest bus, and the ratio falls as the pool grows: 4.8 on the M5 Max's 128 GB, 2.4 on the M5 Ultra's 512, 1.6 on the M3 Ultra's, 2.1 on the Spark's 128 GB, 0.8 on Vera's 1.5 TB. The Spark's 273 GB/s buys a 27B ceiling of 19 tokens per second, a third of the M3 Ultra's and the same class as AMD's 128 GB part; NVIDIA's own pool, at its own consumer scale, decodes slower than the Mac's. That is not a failing of NVIDIA's engineering; it is the physics of a wide bus and a large memory not fitting in one package at once, which is the whole of Apple's homework track. The Vera CPU makes the trade explicit at the other end: "up to 1.5 TB of memory" on "SOCAMM, detachable, field-replaceable modules" at "up to 1.2 terabytes per second" — a terabyte and a half you can swap, at a bandwidth per gigabyte below the Mac's, on a chip that is a CPU beside a GPU rather than a pool for one.

What the Copy Proves

Two things, and the convergence track takes both further. First, that the architecture was right: when the largest GPU company builds a coherent pool from its CPU's LPDDR to its GPU and sells a desktop with 128 GB of it, the diagram Apple drew for a laptop in 2020 has become the industry's diagram for the wall's other side. Second, that the pool is not Apple's alone, and its advantages — capacity, one address space, no copy — are now available with CUDA attached, at a bandwidth-per-gigabyte that the same physics constrains. The Spark's price tells the last part of the story: launched at $3,999, raised to $4,699 "due to the constrained memory supplies worldwide" per NVIDIA's own forums as reported — the same memory market in which the Mac Studio's 512 GB configuration stopped being sold a week later, with Apple silent and the press citing the DRAM shortage. Everyone who builds a pool now buys the same DRAM.

Code

pools.py — every pool on one table, with the ratio that decides decode·python
#!/usr/bin/env python3
"""NVIDIA builds unified memory too. Every pool in one table: capacity, bandwidth, the
ratio between them, and the link that makes it one pool. Vendor figures; the ratio and
the 27B ceiling are divisions."""
pools = [  # name, GB, GB/s, link, source
    ("Apple M3 Ultra, Mac Studio",            512, 819,  "UltraFusion, 'over 2.5TB/s' die to die",           "Apple"),
    ("Apple M5 Ultra, Mac Studio",            512, 1229, "UltraFusion, 'over 4.4TB/s'",                       "Apple"),
    ("NVIDIA GB10, DGX Spark",                128, 273,  "NVLink-C2C, '5x the bandwidth of fifth-generation PCIe'", "NVIDIA"),
    ("NVIDIA GH200 Grace Hopper (CPU+GPU)",   480+96, 0, "NVLink-C2C, '900 gigabytes per second'; 'a single per-process page table'", "NVIDIA"),
    ("NVIDIA Vera CPU (SOCAMM)",              1500, 1200, "LPDDR5X on 'detachable, field-replaceable modules'", "NVIDIA"),
    ("AMD Ryzen AI Max+ 395",                 128, 256,  "one die's memory controller; 256-bit LPDDR5x-8000", "AMD"),
]
print(f"{'pool':38} {'GB':>5} {'GB/s':>5} {'GB/s per GB':>11} {'27B ceiling':>11}   link")
for name, gb, bw, link, src in pools:
    ratio = f"{bw/gb:11.2f}" if bw else f"{'n/a':>11}"
    ceil = f"{bw/14.42:11.0f}" if bw else f"{'n/a':>11}"
    print(f"{name:38} {gb:5d} {bw:5d} {ratio} {ceil}   {link} [{src}]")
print("\nGH200's pool is CPU LPDDR5X plus GPU HBM3 behind one page table; its two halves have different bandwidths, so no single figure is honest.")

# pool                                      GB  GB/s GB/s per GB 27B ceiling   link
# Apple M3 Ultra, Mac Studio               512   819        1.60          57   UltraFusion, 'over 2.5TB/s' die to die
# Apple M5 Ultra, Mac Studio               512  1229        2.40          85   UltraFusion, 'over 4.4TB/s'
# NVIDIA GB10, DGX Spark                   128   273        2.13          19   NVLink-C2C, '5x the bandwidth of fifth-generation PCIe'
# NVIDIA GH200 Grace Hopper (CPU+GPU)      576     0         n/a         n/a   NVLink-C2C, '900 gigabytes per second'; 'a single per-process page table'
# NVIDIA Vera CPU (SOCAMM)                1500  1200        0.80          83   LPDDR5X on 'detachable, field-replaceable modules'
# AMD Ryzen AI Max+ 395                    128   256        2.00          18   one die's memory controller; 256-bit LPDDR5x-8000

External links

Exercise

Add two more pools to pools.py from vendor pages — any machine sold with 'unified' or 'coherent' memory — and compute their bandwidth per gigabyte. Plot the column mentally against capacity and write on your card where each lands relative to the M3 Ultra's 1.6. Then note which one you would buy for the largest model you run, and why the ratio, not the capacity alone, decided it.
Hint
Every pool you add will land on the same curve: more gigabytes, fewer GB/s per GB. If one seems to break the curve, check whether its bandwidth figure is for the GPU's own HBM rather than the shared pool — GH200 is the usual confusion, with two memories behind one page table.

Progress

Progress is local-only — sign in to sync across devices.
Spotted a bug or have feedback on this page?Report an Issue

Comments 0

🔔 Reply notifications (sign in)
Sign inPlease sign in to comment.

No comments yet — be the first.