mlx-lm in Five Minutes — Load, Prompt, Done

Why this lesson is short on purpose

The fastest way to internalize that mlx-lm works is to watch it work. We will install nothing new (you already have mlx-lm from core.lesson1), pull a small instruct model from the trusted mlx-community Hugging Face org, and call generate. Three lines, one cup of coffee, real LLM output on your Mac.

The rest of the track unpacks every piece of what just happened. This lesson is the "wait, that's it?" moment that earns the unpacking.

Pick a small model that demonstrates the loop

For this lesson we use mlx-community/Llama-3.2-1B-Instruct-4bit. Reasons:

Small (~700 MB) — downloads in seconds on a normal connection, fits in any M-series Mac's unified memory.
Instruct-tuned — answers your prompt, doesn't just continue your text. Makes the demo legible.
From mlx-community — already MLX-format, already quantized, already vetted by the community curation we discussed in foundations.lesson6.
4-bit quantized — fits the napkin math from foundations.lesson4 at the smallest reasonable footprint without losing the demo.

The two-liner that does it

The code block below is the entire load-and-generate loop. load() downloads the model on first call (cached after, so subsequent runs are instant) and returns a model + tokenizer pair. generate() takes a prompt and returns the completion as a Python string.

What just happened, briefly

HF download or cache hit — first call pulls from huggingface.co/mlx-community/Llama-3.2-1B-Instruct-4bit into ~/.cache/huggingface/hub/. Subsequent runs read from disk.
Weights map into unified memory — the safetensors shards are mmap'd; MLX kernels read directly from that mapped region.
Generate runs token-by-token — under the hood, generate wraps stream_generate (lesson 2) and concatenates the token texts. Each forward pass is a kernel dispatch on the GPU; the KV cache builds up as you go (lesson 7).

That's the whole pipeline. The next six lessons in this track look at each piece in detail — streaming, sampling, architectures, chat templates, the built-in HTTP server, and the memory behavior. The point of this lesson is to know that the pipeline exists and works before we earn the right to dissect it.

Code

Two-liner — load + generate·python

from mlx_lm import load, generate

model, tokenizer = load("mlx-community/Llama-3.2-1B-Instruct-4bit")
print(generate(model, tokenizer, prompt="Say hello in one short sentence.", max_tokens=30))

# Verified output (2026-05-03, mlx-lm 0.31.3):
#   '"Hello, how are you?" is a simple yet effective greeting that can start
#    a conversation and break the ice.\n\nHere are some more ideas for greetings'
#
# (Your exact output will vary — sampling defaults are non-deterministic
#  unless you pass a fixed seed. We'll fix the determinism story in lesson 3.)

Where the model lives on disk after first load·bash

# After the first `load()` call, the model is cached here:
ls ~/.cache/huggingface/hub/models--mlx-community--Llama-3.2-1B-Instruct-4bit/snapshots/

# Check disk usage:
du -sh ~/.cache/huggingface/hub/models--mlx-community--Llama-3.2-1B-Instruct-4bit/

# Sample (verified 2026-05-03):
#   ~700 MB on disk for the 1B Q4 instruct variant

Exercise

Run the two-liner. Confirm a real completion comes back. Then call generate() three more times with the same model object — note that there's no re-download, no re-load, just a fresh forward pass each time. Time the first generation vs the third with time.perf_counter(); the first one will be slightly slower because of MLX's first-call JIT warmup. Two sentences on what you observed.