Silicon Built for Software: Rosetta's Memory Ordering and the Hidden Matrix Coprocessor

"Some of the transistors on this die exist because of software Apple wanted to run, not software Apple wanted to write."

Three Things an x86 Program Assumes

An Intel-era Mac program was compiled against three facts of the x86 world that Arm does not share. Its memory is arranged in 4 KB pages; Apple silicon's native page is 16 KB. Its loads and stores appear to other cores in a strict order — x86's total store ordering — while Arm's memory model is weakly ordered, so a translated program that assumed the x86 rules would see stale data and race in ways it never did on Intel. And it may lean on x86 vector instructions (SSE, AVX) that have no direct Arm twin. Rosetta 2 translates all three in software; the first two are the assumptions Apple also built silicon for, and the matrix coprocessor later in this lesson exists for Apple's own libraries, not for translation.

Apple's engineers said so on stage in 2020: "Page size, memory ordering rules … all change. For applications running in Rosetta, we've made sure that everything matches behavior on an Intel-based Mac." The two mechanisms behind that sentence are visible from a shell, and one of them is in the open-source kernel.

Pages: 16 KB Natively, 4 KB Under Translation

Run the code block. A native Python reports a page size of 16384; the same Python launched under arch -x86_64 — as a translated x86 process — reports 4096. The kernel gives translated processes the page size they were compiled for, exactly as the WWDC session said: "Native page size is different … 4 kB pages for translated processes." The 16 KB native page is a phone inheritance (fewer page-table walks, less TLB pressure for large working sets) and it surfaces elsewhere in this quest: every fleet Mac reports hw.pagesize 16384, and the kernel's own constant, PAGE_MAX_SHIFT 14, is where the number comes from.

Memory Ordering: A Mode Bit for Intel's Rules

The harder problem was ordering, and Apple solved it in hardware rather than by making the translator insert fences everywhere (which would have made translated code slow). Apple's cores have a mode in which they enforce x86-style total store ordering, and the kernel switches it on for translated threads. Apple has never documented this by name; the evidence is the kernel source, where the control-register bit is defined as ACTLR_EL1_EnTSO. Label that carefully: a fact read from open-source XNU, corroborated by Apple's stage sentence, not an Apple-documented feature. It is also the cleanest example in the whole chip of the industry-level argument from track two — Apple could afford to spend transistors on a competitor's memory model because it owns the core, the kernel and the translator together.

The Matrix Coprocessor Apple Never Named

Since the M1, Apple's chips have carried matrix-multiply units on the CPU side that Apple has never given a public name. The 2020 session called them "matrix multiplication machine learning accelerators" that developers could "leverage more directly using the Accelerate framework"; the community that reverse-engineered them calls the unit AMX, and its research covers M1 through M4 Max. The documented, architectural successor is Arm's Scalable Matrix Extension, which the M4 generation implements — the FEAT_SME flag from lesson one — with the kernel's own header confirming it and a 2024 paper, "Hello SME!", measuring it. On the M3 Ultra the flag is 0 and the undocumented unit is what Accelerate uses.

How much is it worth? The code block measures a 4096×4096 single-precision matrix multiply three ways on office: NumPy through Accelerate (which reaches the matrix units), MLX on the CPU stream (same units, a different library), and MLX on the GPU. Best of five runs, the CPU path lands at about 5 TFLOP/s; the GPU at about 19. That is the honest scale of the hidden coprocessor: roughly a quarter of the GPU's matrix throughput, from the CPU, with no Metal involved. It is why Accelerate-backed code is fast on a Mac, why a CPU fallback in an inference runtime is not the disaster it is elsewhere, and why the GPU is still the right place for a language model — the GPU has the bandwidth, and the coprocessor does not change that.

Code

pages.sh — the page size a process sees depends on what it is·bash

# native arm64 process
python3 -c "import os, platform; print(os.sysconf('SC_PAGESIZE'), platform.machine())"
# 16384 arm64

# the same interpreter launched as a translated x86-64 process (Rosetta 2 must be installed)
arch -x86_64 /usr/bin/python3 -c "import os, platform; print(os.sysconf('SC_PAGESIZE'), platform.machine())"
# 4096 x86_64                  <- office, M3 Ultra, macOS 26.6.2, 2026-09-15

sysctl hw.pagesize            # 16384 on every Apple silicon Mac
# The kernel's constant: PAGE_MAX_SHIFT 14 in XNU's arm64 headers (2^14 = 16384).
# The ordering mode Rosetta relies on: ACTLR_EL1_EnTSO in osfmk/arm64/proc_reg.h.

matmul_three_ways.py — the CPU's matrix units against the GPU, one machine·python

#!/usr/bin/env python3
"""4096x4096 float32 matmul: NumPy via Accelerate (CPU matrix units), MLX on the
CPU stream, MLX on the GPU. TFLOP/s = 2·n³ / seconds, best of five. Run in an env
whose NumPy links Accelerate (numpy.show_config() names it) with mlx installed."""
import time
import numpy as np
import mlx.core as mx

n = 4096
a = np.random.rand(n, n).astype(np.float32)
b = np.random.rand(n, n).astype(np.float32)


def best_of(fn, runs: int = 5) -> float:
    fn()                                          # warm
    return min(timed(fn) for _ in range(runs))


def timed(fn) -> float:
    t = time.perf_counter(); fn(); return time.perf_counter() - t


def tflops(seconds: float) -> float:
    return 2 * n**3 / seconds / 1e12


cpu_np = best_of(lambda: a @ b)

with mx.stream(mx.cpu):
    x, y = mx.array(a), mx.array(b)
    cpu_mx = best_of(lambda: mx.eval(x @ y))

x, y = mx.array(a), mx.array(b)
gpu_mx = best_of(lambda: mx.eval(x @ y))

print(f"NumPy / Accelerate (CPU matrix units): {tflops(cpu_np):5.2f} TFLOP/s")
print(f"MLX cpu stream:                        {tflops(cpu_mx):5.2f} TFLOP/s")
print(f"MLX gpu (Metal):                       {tflops(gpu_mx):5.2f} TFLOP/s")
# office, M3 Ultra, mlx 0.32.2, numpy (accelerate), 2026-09-15, best of five:
# 4.92 / 5.05 / 19.53 TFLOP/s

Exercise

Run pages.sh (install Rosetta with softwareupdate --install-rosetta if the arch command fails) and matmul_three_ways.py on your Mac. Add the three TFLOP/s figures and the page sizes to your card. Then answer: for a language model whose decode is bandwidth-bound, does the CPU's matrix throughput change the decode ceiling at all? Explain in terms of which resource the ceiling formula divides by.

Hint

The ceiling is bandwidth ÷ bytes per token; no compute unit appears in it. The CPU matrix units matter for prefill on machines without a usable GPU and for everything that is compute-bound; they do not add a single byte per second to the memory bus.