MLX-format vs GGUF — When to Pick Which

Two formats, two heritages

If you've used llama.cpp or Ollama, you've seen GGUF — a single binary file containing weights, config, tokenizer, and metadata. If you've used mlx-lm, you've seen the MLX format — a directory of standard files (covered in lesson 1). Both formats serve the same general purpose: ship a quantized LLM ready for inference. They make different trade-offs, and which one you pick depends on where you're deploying.

The shape of each format

GGUF is a single self-contained binary. Weights, tokenizer, config — all packaged inside. Designed for cross-platform CPU and GPU inference (it grew up in llama.cpp's CPU world). Easy to ship — one file, copy and run. Hard to inspect or modify — you need GGUF-aware tools to read what's inside. Optimized for the loaders that ship with llama.cpp, Ollama, LM Studio, and the rest of that ecosystem.

MLX format is a directory of standard files (safetensors + JSON configs + tokenizer files). Designed for Apple Silicon's unified-memory hardware. Files map directly into MLX's expected memory layout, no translation needed at load time. Easy to inspect or hack — open in any editor. Heavier to ship — multiple files, a few hundred kilobytes of metadata.

Where each one wins

You're deploying inside the Apple-only ecosystem (mlx-lm, mlx-vlm, mlx-audio, MLX Swift, Ollama on Mac since v0.19) → MLX format. Native dispatch, no translation overhead, the dispatcher reads the standard configs directly.
You're handing the binary to a heterogeneous team with mixed Mac / Linux / Windows machines → GGUF. One file, runs in llama.cpp on any platform, runs in Ollama on any platform.
You want to fine-tune or modify the model → MLX format. The directory layout makes it easy to swap layers, modify configs, or merge adapters (Track 4 covers this).
You're at the size frontier (very small chip, very strict memory) → GGUF historically had a slight edge for the lowest-bit quantizations on CPU. With MLX's recent quantization work the gap has narrowed; for Mac-native use, MLX wins on the GPU pathway.
You're running Ollama on Mac → either format works; Ollama since v0.19 is built on MLX, so MLX-format is the native path. GGUF still works as a legacy import.

Conversion between the two

You can convert MLX → GGUF or GGUF → MLX, but it's not a one-line command — both directions require intermediate tools (typically going through HF transformers' representation). For most workflows, you decide upfront which format you want and convert from the upstream PyTorch source directly. Cross-format conversion is for the edge case where you have one format and need the other.

The non-decision

For Mac-only Apple Silicon work — which is where this whole quest lives — MLX format is the default. You shouldn't think about GGUF unless you're actively shipping to non-Apple deployment targets. The lesson exists so you can recognize when GGUF is the right call (heterogeneous teams, llama.cpp-based pipelines), not so you spend time agonizing over the choice for your local Mac work.

Code

Spot the format difference on disk·bash

# MLX-format model: directory of standard files
ls ~/.cache/huggingface/hub/models--mlx-community--Llama-3.2-1B-Instruct-4bit/snapshots/*/
# config.json  model.safetensors  tokenizer.json  tokenizer_config.json  ...

# GGUF model: single binary file (path varies by uploader)
find ~/.cache/huggingface/hub -name "*.gguf" 2>/dev/null | head -3
# /Users/.../models--Qwen--Qwen2-0.5B-Instruct-GGUF/snapshots/.../qwen2-0_5b-instruct-q8_0.gguf

Inspect a GGUF file's metadata without loading the weights·python

# Requires `pip install gguf` — it's a small package separate from llama.cpp.
# (Don't install in your `mlx` env unless you want it; this is illustrative.)
from gguf import GGUFReader

# Path to any .gguf file you have
path = "/path/to/some-model.gguf"
reader = GGUFReader(path)

print(f"Architecture: {reader.fields.get('general.architecture').contents()}")
print(f"Quantization: {reader.fields.get('general.file_type').contents()}")
print(f"Tensors:      {len(reader.tensors)}")

Exercise

If you have any GGUF model on your machine (or grab a small one from HF — many models are uploaded in both formats), inspect both the GGUF and MLX versions of the same model side by side. Note the file structure difference (single binary vs directory of files), the disk size difference (usually similar within ~10%), and the load-time difference (MLX is typically faster to mmap on Apple Silicon). Two sentences on which format feels more natural for your typical workflow.