Performance per Watt Was the Point

"Every design decision in this track was made under one constraint. Not speed. Not cost. Heat."

The Constraint That Explains the Track

Read the previous five lessons again with one question: why? Why an architecture license instead of a bought core — so the core could be tuned for a power budget. Why wide instead of fast — because the last gigahertz costs more energy than the first. Why two kinds of core — so background work runs on the cheap ones. Why a matrix unit on the CPU and a Neural Engine and a media engine — because a specialist does its job at a fraction of a general core's energy. Every answer is the same answer: the chip was designed for a device with a battery and no fan, where performance per watt is not a marketing line but the only metric that lets the product exist.

When the design moved to the Mac, the constraint relaxed and the shape stayed. A Mac Studio can draw far more than a phone, and Apple's specification sets its "maximum continuous power" at 480 W — but the chip inside was never built to use that, and Apple's own measurements say it does not: the 512 GB M3 Ultra configuration draws 9 W idle and 270 W at maximum in Apple's test. The rest of the power budget is headroom the design does not spend.

What This Means for a Machine That Runs Inference All Day

The household's Macs are on 24 hours a day. Several of them serve inference continuously. Two numbers from Apple's power page set the scale of that:

Machine (Apple's tested configuration)	Idle	Maximum	Evidence
Mac Studio (2025), M3 Ultra 32C/80G, 512 GB	9 W	270 W	vendor (support.apple.com/102027)
Mac Studio (2025), M4 Max 14C/32G, 36 GB	6 W	145 W	vendor (same page)
Mac Studio, maximum continuous power (spec)	—	480 W	vendor (tech specs)
GeForce RTX 5090 (the card alone)	—	575 W	vendor (NVIDIA)
RTX PRO 6000 Blackwell (the card alone)	—	600 W	vendor (NVIDIA datasheet)
DGX Spark (GB10 SoC TDP; the box ships a 240 W supply)	—	140 W	vendor (NVIDIA hardware page)

An M3 Ultra that decodes a 27B model at 32.6 tokens per second (measured, office, lab track) cannot be drawing more than 270 W while it does so, and is almost certainly drawing much less, since Apple's maximum is a whole-machine stress figure. That puts an upper bound on the energy per token at about 8 joules, and the true figure below it. The code block does that arithmetic and shows how to replace the bound with a measurement: powermetrics, Apple's own sampler, reports CPU, GPU and Neural Engine power in milliwatts, and needs only root and a model to run. This quest's session could not run it — the lab machines are reached over ssh without a password for root — so the lesson gives you the command and the bound, and labels the bound as a bound.

The Honest Comparison

A discrete card wins the absolute decode race on any model that fits in its memory (the CUDA track). It does so at a power budget the Mac never approaches: a 575 W card in a machine with its own processor and memory is a kilowatt-class computer, and it is loud. The Mac's position is a different point on the curve — fewer tokens per second at under half the card's power, silent, idling at 9 W, and able to hold a model the card cannot. What these bounds do not hand the Mac is fewer joules per token: at the vendor maxima the card's 4.6 J per token beats the Studio's 8.3 J bound, and where the Mac actually sits below its bound is a powermetrics reading this quest could not take. Neither point is the right one for everyone. The edge-era track argues which workloads belong at which point; this lesson's job is only to show that the two points exist because two different constraints drew them, and that the Mac's was heat.

Code

energy_per_token.py — bounds from vendor power figures and measured decode·python

#!/usr/bin/env python3
"""Joules per token, bounded. Power figures are vendor maxima (Apple's tested
maximum for the exact office configuration; NVIDIA board power). Decode rates
are measured (office) or ceiling arithmetic (the card) — the labels say which."""

rows = [
    # machine,                        watts, tok/s,  power label,           rate label
    ("office M3 Ultra, Qwen3.5-27B 4-bit", 270, 32.6, "vendor max (Apple)",   "measured 2026-09-15"),
    ("office M3 Ultra, Qwen3.5-9B 4-bit",  270, 95.1, "vendor max (Apple)",   "measured 2026-09-15"),
    ("RTX 5090, 14.4 GB model (fits)",     575, 1792 / 14.4, "vendor board power", "ceiling = 1792 GB/s ÷ 14.4 GB (physics, not measured)"),
]

print(f"{'machine / model':38} {'W':>4} {'tok/s':>6} {'J/token (upper bound)':>22}")
for name, w, tps, plabel, rlabel in rows:
    print(f"{name:38} {w:4d} {tps:6.1f} {w / tps:22.2f}   [{plabel}; {rlabel}]")

print("\nA bound is not a measurement. Replace the watts with a powermetrics sample:")
print("  sudo powermetrics --samplers cpu_power,gpu_power,ane_power -i 1000 -n 30")
print("while a model decodes; add CPU + GPU power, divide by tokens per second.")

Measure it yourself: powermetrics while a model decodes (needs root)·bash

# terminal 1: decode something for ~30 s in your MLX env
python -m mlx_lm generate --model mlx-community/Qwen3.5-9B-4bit \
  --prompt "Explain unified memory in 400 words." --max-tokens 300

# terminal 2: sample power once a second for 30 s
sudo powermetrics --samplers cpu_power,gpu_power,ane_power -i 1000 -n 30 \
  | grep -E "CPU Power|GPU Power|ANE Power"
# CPU Power: 1234 mW
# GPU Power: 45678 mW      <- the number that matters for decode
# ANE Power: 0 mW          <- the Neural Engine is idle; MLX runs on the GPU

# joules per token = (CPU mW + GPU mW) / 1000 / tokens-per-second

Exercise

Run the powermetrics recipe on your Mac while a model decodes for thirty seconds. Record the mean GPU and CPU power and the decode rate, compute joules per token, and add it to your card as a measured row with the full stamp. Then compare it to the bound in energy_per_token.py and write one sentence on how far below Apple's maximum your Mac actually ran.

Hint

Expect the GPU rail to carry nearly all of it and the ANE rail to read zero. On a laptop, run it plugged in — on battery the scheduler may cap the GPU and you would be measuring a policy, not the chip.