Memory Optimization — Where the Mac's Wall Actually Is

Where the Mac's wall actually is

The napkin math from foundations.lesson4 tells you roughly how much memory a model needs. The reality is that macOS doesn't let your GPU lock 100% of unified memory — there's a wired-memory ceiling that varies by Mac SKU, typically reserving ~25% of total RAM for OS use and other apps. On a 512 GB M3 Ultra Studio, the practical GPU ceiling is closer to 380 GB; on a 64 GB MacBook Pro, closer to 48 GB.

This lesson is the diagnostic toolkit for figuring out where your wall is and what to do when you hit it.

The diagnostic commands

Three sysctl values tell you everything you need to know about the wired-memory situation:

hw.memsize — total physical unified memory in bytes.
iogpu.wired_limit_mb — explicit GPU wired-memory cap in MB. 0 means "system default" (typically ~75% of total).
vm.memory_pressure (via memory_pressure tool) — current pressure state (normal, warn, critical).

Raising the ceiling — when it's safe

You can raise iogpu.wired_limit_mb via sudo sysctl iogpu.wired_limit_mb=N to give the GPU more memory than the default. Whether this is safe depends on what else you're running:

Mac Studio with 192-512 GB dedicated to ML work, no other heavy apps — raising to 90% of total is reasonable.
MacBook with 16-64 GB running an editor, browser, and other apps — DON'T raise it. macOS will start swapping or killing processes if the OS doesn't get its share.
If you raise it temporarily for one large job, set it back when done — the value persists across the session but resets at reboot.

What to do when you hit the wall

Drop quantization — Q4 → Q4 with smaller group_size, or Q4 → mixed_3_4 (Track 3). Each step shaves real memory.
Shorten context — KV cache scales linearly with sequence length. If you're seeing OOM during long-context generation, cap max_tokens or use a smaller-context model.
Shrink the model — drop one tier (70B → 13B) before fighting OS-level memory limits. Cheaper than raising sysctl.
Use a smaller base model + LoRA instead of a larger model in full precision. Track 4's lessons apply here too.

The peak-memory measurement workflow

Whenever you commit a model + workload to production, measure peak memory once. mx.get_peak_memory() at the end of a representative inference run gives you the actual ceiling. Add 20% safety margin, ensure that's still under your wired-memory cap, and you're calibrated.

Code

Diagnose the unified-memory state on your Mac·bash

# Total unified memory
sysctl -n hw.memsize | awk '{ printf "Total RAM         : %.1f GB\n", $1/1024/1024/1024 }'

# GPU wired-memory cap (0 = system default ~ 75% of total)
sysctl iogpu.wired_limit_mb

# Current pressure state
memory_pressure

# Sample (M3 Ultra Studio):
#   Total RAM         : 512.0 GB
#   iogpu.wired_limit_mb: 0
#   The system has X% of memory free.

Measure peak memory for an inference run·python

import mlx.core as mx
from mlx_lm import load, generate

mx.reset_peak_memory()
model, tok = load("mlx-community/Llama-3.2-1B-Instruct-4bit")
print(f"After load             : peak {mx.get_peak_memory()/1024/1024:.1f} MB")

generate(model, tok, prompt="Tell a long story:", max_tokens=200, verbose=False)
print(f"After 200-tok generate : peak {mx.get_peak_memory()/1024/1024:.1f} MB")

# Add a 20% safety margin to the peak before declaring "this fits".
# Verified (M3 Ultra Studio):
#   After load             : peak ~675 MB
#   After 200-tok generate : peak ~685 MB

Raise the ceiling temporarily (only on dedicated ML Macs)·bash

# Raise GPU wired memory to 460 GB (only on a 512 GB Studio used for ML work)
sudo sysctl iogpu.wired_limit_mb=460000

# Verify
sysctl iogpu.wired_limit_mb

# Reset to system default when done (or just reboot)
sudo sysctl iogpu.wired_limit_mb=0

# WARNING: Do NOT do this on a MacBook you also use for daily work.
# macOS needs its share for the OS itself, kernel buffers, and other apps.

Exercise

Run the diagnostic block on your Mac. Note your total RAM, current wired_limit, and memory_pressure state. Then run the peak-memory measurement on a model you actually use. Compute (peak + 20%) / wired_cap — that's your headroom utilization. If it's above 80%, you're close to the wall and the next bigger model won't fit. Two sentences on whether you're safe or near the ceiling.