The MLX Model Format — safetensors + config + tokenizer

What lives in an MLX model directory

Every MLX-format model on Hugging Face (or on your disk after mlx_lm.convert) is a small set of files in a directory. There's no opaque blob, no proprietary container — it's all readable formats you can inspect with standard tools.

model.safetensors (or sharded — model-00001-of-00003.safetensors etc.) — the weights, in safetensors format. mmap-able, deduplicated by tensor name, fast to load.
model.safetensors.index.json — only present when sharded; maps each tensor name to which shard file holds it.
config.json — the architecture descriptor. model_type, hidden size, layer count, attention head count, vocab size, and quantization config (if quantized). This is what mlx-lm reads to dispatch to the right model class.
tokenizer.json + tokenizer_config.json — the tokenizer, its special tokens, and (importantly) its chat_template as a Jinja string. Lesson 5 of the lm track depends on these.
Optional — generation_config.json with default sampling params, special_tokens_map.json, added_tokens.json, etc. mlx-lm reads what it needs and ignores the rest.

Why this matters

Three reasons. First, you can verify what you have. cat config.json tells you the architecture and quantization at a glance — no need to guess from a model card. Second, you can reuse pieces. The same tokenizer files work across model variants of the same family; you can swap weights without re-bundling tokenization. Third, you can debug load failures. "Weight key not found" errors usually mean the safetensors and the config disagree — open both files and look.

Sharded vs single-file

Small models (under ~5 GB on disk) ship as a single model.safetensors. Larger models are split into shards, with model.safetensors.index.json as the routing map. mlx-lm handles both transparently — you don't pick the shard, the loader does. The shard size limit is set during conversion (typically 5 GB per shard).

What you do with this knowledge

For the rest of this track, every lesson assumes you can ls a model directory and recognize what's there. Get comfortable opening config.json for any model you're about to load — it answers "what architecture, what dtype, what quantization, what context length" in 30 seconds.

Code

Inspect a cached MLX model directory·bash

# Find the cached model
SNAP=~/.cache/huggingface/hub/models--mlx-community--Llama-3.2-1B-Instruct-4bit/snapshots/
ls $SNAP*/

# Sample listing (verified 2026-05-03):
#   config.json
#   model.safetensors
#   model.safetensors.index.json   (only if sharded; 1B Q4 fits in single file)
#   special_tokens_map.json
#   tokenizer.json
#   tokenizer_config.json

Read the config.json — architecture + quantization at a glance·python

import json, os, glob

snap = glob.glob(os.path.expanduser(
    "~/.cache/huggingface/hub/models--mlx-community--Llama-3.2-1B-Instruct-4bit/snapshots/*/"
))[0]

with open(os.path.join(snap, "config.json")) as f:
    cfg = json.load(f)

print("model_type      :", cfg.get("model_type"))
print("hidden_size     :", cfg.get("hidden_size"))
print("num_hidden_layers:", cfg.get("num_hidden_layers"))
print("num_attention_heads:", cfg.get("num_attention_heads"))
print("vocab_size      :", cfg.get("vocab_size"))
print("max_position_embeddings:", cfg.get("max_position_embeddings"))
print("quantization    :", cfg.get("quantization"))   # group_size, bits, mode

Peek at safetensors metadata without loading the weights·python

import os, glob
from safetensors import safe_open

snap = glob.glob(os.path.expanduser(
    "~/.cache/huggingface/hub/models--mlx-community--Llama-3.2-1B-Instruct-4bit/snapshots/*/"
))[0]
shard = os.path.join(snap, "model.safetensors")

with safe_open(shard, framework="numpy") as f:
    keys = list(f.keys())
    print(f"Total tensors: {len(keys)}")
    print("First 8 tensor names:")
    for k in keys[:8]:
        print(f"  {k:60} dtype={f.get_slice(k).get_dtype()} shape={f.get_slice(k).get_shape()}")

Exercise

Inspect any cached MLX model on your machine. Read its config.json and write down (1) the architecture name, (2) the quantization config (group_size, bits, mode), (3) the max context length. Then open tokenizer_config.json and find the chat_template field — copy the first 200 characters. The exercise is to retire the "what's in this thing?" anxiety: a 30-second inspection answers it for any MLX model.