The Community Finds the Mac

"Two weeks after Meta announced the weights and one week after they leaked out into the world, someone had them running on a MacBook. Apple's framework was nine months away."

February to March 2023

On 2023-02-24 Meta announced LLaMA, a family of language models from 7 to 65 billion parameters, released to researchers — and, within days, to everyone. It was the first model of that class whose weights a person could hold. On 2023-03-10 a repository called llama.cpp appeared: a C++ implementation whose stated purpose was, and still is, "LLM inference in C/C++", and whose README declares "Apple silicon is a first-class citizen — optimized via ARM NEON, Accelerate and Metal frameworks." It ran on the CPU first, using the Accelerate framework's matrix routines — the CPU track's hidden coprocessor, reached exactly the way Apple said it could be. On 2023-06-04 the repository merged "llama : Metal inference", and from that day the model's decode ran on the Apple GPU, reading weights from the pool the GPU shared with everything else.

Two things about that sequence are the point of this lesson. First, it was fast: a hundred days from ChatGPT to a runnable open model on a laptop, and fourteen from the announcement to the repository — seven from the leak. Second, it was not Apple. Apple's own framework, MLX, appeared on 2023-11-28 — 263 days after llama.cpp — and its first release on PyPI a week later. The community found the Mac before Apple did, using Apple's public doors (Accelerate, then Metal) and Apple's memory layout, and reported back what it found: a laptop that could hold a 65B model and decode it at a usable rate, when no consumer graphics card could hold it at all.

What They Found, in the Physics Track's Terms

What the community discovered was not a trick; it was the memory track and the physics track, empirically. A 4-bit 65B model is about 39 GB; a 64 GB M1 Max holds it, a 24 GB card does not. Decode is bandwidth-bound; the M1 Max's 400 GB/s over 39 GB gives a ceiling near ten tokens per second, which is reading speed. The quantized formats llama.cpp invented — its GGML and later GGUF files with 4-bit weights and per-block scales — are the reason "fits" applied at all, and the reason this quest's ladder runs at 4 bits. None of it required anything Apple had not already shipped. It required someone to notice that the shape built for ProRes was the shape a model wanted.

Run the Community's Runtime on the Fleet

llama.cpp is still a first-class door to the GPU (the GPU track's table), and this quest builds it from source on office to keep the lesson honest — the code block is the build, with Metal enabled, and the one command that proves the GPU backend is present. The lab's numbers are from MLX because MLX is what the household runs; llama.cpp on the same machine is the exercise, and its llama-bench reports the same two phases this quest has measured throughout: prompt processing and token generation.

Code

Build the community's runtime with the Metal backend, and prove the door is open·bash

# office, 2026-09-15 — llama.cpp at commit 4c9233c (that day's master); Xcode 26.6; Homebrew cmake
git clone --depth 1 https://github.com/ggml-org/llama.cpp.git && cd llama.cpp
cmake -B build -DGGML_METAL=ON
cmake --build build --config Release -j 16 --target llama-bench llama-cli

# the GGUF model: llama.cpp's own format; a 4-bit Qwen3.5-9B is a ~5.7 GB download
hf download unsloth/Qwen3.5-9B-GGUF --include "*Q4_K_M*" --local-dir models/

# the benchmark: pp = prompt processing (prefill), tg = token generation (decode)
./build/bin/llama-bench -m models/Qwen3.5-9B-Q4_K_M.gguf -p 512 -n 128 -ngl 99
# office, M3 Ultra, build 4c9233c, 2026-09-15:
# | model                   |     size | params | backend  | test  |            t/s |
# | qwen35 9B Q4_K - Medium | 5.28 GiB | 8.95 B | MTL,BLAS | pp512 | 1175.85 ± 17.04 |
# | qwen35 9B Q4_K - Medium | 5.28 GiB | 8.95 B | MTL,BLAS | tg128 |    69.82 ± 0.42 |
# 'MTL' in the backend column is the whole point of this lesson.
# For scale: mlx-lm 0.31.3 on the same Mac and the same model family decoded at 95 tok/s (4-bit g64).

days.py — the intervals that make the argument·python

#!/usr/bin/env python3
from datetime import date

chatgpt   = date(2022, 11, 30)   # OpenAI
llama     = date(2023,  2, 24)   # Meta: LLaMA announced (gated researcher release)
leak      = date(2023,  3,  3)   # the weights leak (press coverage that week)
llama_cpp = date(2023,  3, 10)   # GitHub: repository created
metal     = date(2023,  6,  4)   # GitHub: 'llama : Metal inference' merged (#1642)
mlx       = date(2023, 11, 28)   # GitHub: MLX repository created

print(f"ChatGPT -> open weights (LLaMA):        {(llama - chatgpt).days:4d} days")
print(f"LLaMA announced -> llama.cpp:           {(llama_cpp - llama).days:4d} days")
print(f"weights leaked -> llama.cpp:            {(llama_cpp - leak).days:4d} days")
print(f"llama.cpp -> decode on the Apple GPU:   {(metal - llama_cpp).days:4d} days")
print(f"community runtime -> Apple's framework: {(mlx - llama_cpp).days:4d} days")

Exercise

Build llama.cpp on your Mac with the Metal backend, download a 4-bit GGUF of a model you already have in MLX form, and run llama-bench with -p 512 -n 128. Put its pp and tg figures beside your MLX ladder numbers on your card. Then write one sentence on why two runtimes reading the same weights from the same pool can differ, and which physics-track lesson explains the difference.

Hint

Same bytes per token, same bus — so the difference is per-token overhead and kernel efficiency, the fixed term and the effective-bandwidth term of the decode-ceiling lesson. Whichever runtime reads each weight exactly once at full width wins decode; whichever launches fewer kernels wins the small models.