Memory Optimization — Where the Mac's Wall Actually Is · MLX Quest

Where the Mac's wall actually is

The napkin math from foundations.lesson4 tells you roughly how much memory a model needs. The reality is that macOS doesn't let your GPU lock 100% of unified memory. Metal reports a ceiling for every Mac — recommendedMaxWorkingSetSize, which MLX exposes as max_recommended_working_set_size and treats as the system wired limit — and Apple documents no fixed fraction for it. So don't carry a percentage around; read the number. Here is what nine Macs report on macOS 26.6.2 with the default setting (GB here means 1024³ bytes, the same unit as the memory size on the box):

Unified memory	GPU ceiling	Share	Largest single buffer
24 GB (M3)	17.8 GB	74%	13.3 GB
64 GB (M4 Pro)	51.8 GB	81%	38.9 GB
128 GB (M3 Max, M4 Max, M5 Max)	107.5 GB	84%	80.6 GB
192 GB (M2 Ultra)	161.3 GB	84%	121.0 GB
512 GB (M3 Ultra)	464.0 GB	90.6%	348.0 GB

Two things stand out. The share follows the amount of memory, not the chip: three generations of Max chips with 128 GB report byte-identical ceilings, and the share climbs from 74% at 24 GB to 90.6% at 512 GB. And the largest single buffer is a separate, lower limit — one array can't exceed it even when the total would fit.

It is a soft wall, not a hard one. Apple defines the number as an approximation of how much the GPU can allocate without affecting its runtime performance. MLX lets you allocate past it — its own memory limit defaults to 1.5 times this ceiling — but you pay in speed, and mlx-lm prints a warning once a model's weights pass 90% of it.

This lesson is the diagnostic toolkit for figuring out where your wall is and what to do when you hit it.

The diagnostic commands

Four readings tell you where your wall is:

hw.memsize — total physical unified memory in bytes.
iogpu.wired_limit_mb — explicit GPU wired-memory cap in MB. 0 means the system default from the table above. Any other value becomes the ceiling exactly: set 100000 on a 128 GB Mac and Metal reports 100,000 MB, below that Mac's default of about 110,100 MB. Lower counts as much as higher.
mx.device_info() — MLX's view, in bytes. max_recommended_working_set_size is the ceiling itself and max_buffer_length the largest single buffer. Only the first follows the sysctl; the buffer limit stayed put when the cap moved. The older mx.metal.device_info() still answers but is deprecated.
vm.memory_pressure (via memory_pressure tool) — current pressure state (normal, warn, critical).

Raising the ceiling — when it's safe

You can raise iogpu.wired_limit_mb via sudo sysctl iogpu.wired_limit_mb=N to give the GPU more memory than the default. Whether this is safe depends on what else you're running:

Mac Studio with 192-512 GB dedicated to ML work, no other heavy apps — read the ceiling it already has first (84% at 192 GB, 90.6% at 512 GB). Raise it only when a model's measured peak needs more, and choose a value above that peak but below the machine's total memory, which is the rule the mlx-lm README gives. A value below the default ceiling lowers it.
MacBook with 16-64 GB running an editor, browser, and other apps — DON'T raise it. macOS will start swapping or killing processes if the OS doesn't get its share.
If you raise it temporarily for one large job, set it back when done — the value persists across the session but resets at reboot.

What to do when you hit the wall

Drop quantization — Q4 → Q4 with a larger group_size, or Q4 → mixed_3_4 (Track 4). A larger group stores fewer scales and biases; a lower average bit width shrinks the weights themselves.
Shorten context — KV cache scales linearly with sequence length. If you're seeing OOM during long-context generation, cap max_tokens or use a smaller-context model.
Shrink the model — drop one tier (70B → 13B) before fighting OS-level memory limits. Cheaper than raising sysctl.
Use a smaller base model + LoRA instead of a larger model in full precision. Track 5's lessons apply here too.

The peak-memory measurement workflow

Whenever you commit a model + workload to production, measure peak memory once. mx.get_peak_memory() at the end of a representative inference run gives you the actual ceiling. Add 20% safety margin, ensure that's still under your wired-memory cap, and you're calibrated.

Code

Diagnose the unified-memory state on your Mac·bash

# Total unified memory
sysctl -n hw.memsize | awk '{ printf "Total RAM         : %.1f GB\n", $1/1024/1024/1024 }'

# GPU wired-memory cap (0 = system default, a share that grows with memory size)
sysctl iogpu.wired_limit_mb

# The ceiling itself, as MLX reports it (run inside your mlx env)
python -c 'import mlx.core as mx; i = mx.device_info(); print("GPU ceiling       : %.1f GB" % (i["max_recommended_working_set_size"] / 1024**3)); print("Largest buffer    : %.1f GB" % (i["max_buffer_length"] / 1024**3))'

# Current pressure state
memory_pressure | tail -1

# Verified (2026-09-15, M3 Ultra Studio, macOS 26.6.2, mlx 0.32.2):
#   Total RAM         : 512.0 GB
#   iogpu.wired_limit_mb: 0
#   GPU ceiling       : 464.0 GB
#   Largest buffer    : 348.0 GB
#   System-wide memory free percentage: 94%

Measure peak memory for an inference run·python

import mlx.core as mx
from mlx_lm import load, generate

mx.reset_peak_memory()
model, tok = load("mlx-community/Llama-3.2-1B-Instruct-4bit")
print(f"After load             : peak {mx.get_peak_memory()/1024/1024:.1f} MB")

generate(model, tok, prompt="Tell a long story:", max_tokens=200, verbose=False)
print(f"After 200-tok generate : peak {mx.get_peak_memory()/1024/1024:.1f} MB")

# Add a 20% safety margin to the peak before declaring "this fits".
# Verified (M3 Ultra Studio):
#   After load             : peak ~675 MB
#   After 200-tok generate : peak ~685 MB

Raise the ceiling temporarily (only on dedicated ML Macs)·bash

# Read the current ceiling first. A 512 GB Studio on macOS 26.6.2 already
# reports 475136 MB (464 GB) by default, so any value below that LOWERS it.
python -c 'import mlx.core as mx; print(mx.device_info()["max_recommended_working_set_size"] // 2**20, "MB")'

# Raise GPU wired memory to 480 GB (only on a 512 GB Studio used for ML work,
# and only when a model's measured peak needs more than the default)
sudo sysctl iogpu.wired_limit_mb=491520

# Verify: the MLX reading now matches the sysctl value exactly
sysctl iogpu.wired_limit_mb
python -c 'import mlx.core as mx; print(mx.device_info()["max_recommended_working_set_size"] // 2**20, "MB")'

# Reset to system default when done (or just reboot)
sudo sysctl iogpu.wired_limit_mb=0

# WARNING: Do NOT do this on a MacBook you also use for daily work.
# macOS needs its share for the OS itself, kernel buffers, and other apps.

Exercise

Run the diagnostic block on your Mac. Note your total RAM, current wired_limit, and memory_pressure state. Then run the peak-memory measurement on a model you actually use. Compute (peak + 20%) / wired_cap — that's your headroom utilization. If it's above 80%, you're close to the wall and the next bigger model won't fit. Two sentences on whether you're safe or near the ceiling.

Memory Optimization — Where the Mac's Wall Actually Is