Training Config — Learning Rates, Ranks, Epochs, What Each Lever Does

The five knobs you'll always touch

mlx_lm.lora exposes a long list of CLI flags, but for 90% of fine-tuning runs you'll only adjust a handful. Memorize these five — what they mean, what default values look like, when each is the bottleneck — and you can run a meaningful fine-tuning experiment in minutes.

--num-layers — how many of the model's transformer layers get LoRA adapters attached. Default 16. Setting -1 attaches LoRA to all layers (more capacity, more memory). Higher is more flexible; lower is faster and cheaper.
--learning-rate — the Adam learning rate. Defaults are sane (around 1e-5 for Adam on a 7B base); raise it for faster convergence on small datasets, lower it if loss is unstable. The most common mistake is leaving it at the default when your dataset is tiny.
--batch-size — minibatch size in number of examples per step. Higher means more stable gradients but more memory; lower means less memory and noisier gradients. On a 7B base with a 32 GB Mac, batch sizes of 4–16 are typical.
--iters — total training steps. For a 200–500-example dataset, 100–500 iters is a reasonable starting range. Watch the validation loss; stop when it stops improving.
--max-seq-length — truncates training examples longer than this. Huge impact on memory (KV cache scales with this). Set just above your longest expected example to avoid both truncation losses and wasted memory.

The fine-tune-type knob — pick once, ignore

--fine-tune-type picks between lora (the default), dora (DoRA — see lesson 4), and full (full fine-tuning, almost never the right call on a Mac for a 7B+ model).

Sane defaults you can copy

For a 7B base instruct model, a 200-example domain dataset, and a 32 GB Mac:

--num-layers 16
--learning-rate 5e-5 (raised from default for small dataset)
--batch-size 4
--iters 200
--max-seq-length 2048 (or shorter if your examples are shorter)
--steps-per-eval 25 (val loss every 25 steps)
--save-every 100 (checkpoint every 100 steps)

Watch the loss, not the iters

The right number of iterations isn't a number; it's whatever count makes your validation loss stop improving. mlx-lm reports both training and validation loss at intervals — when validation loss plateaus or starts climbing while training loss keeps falling, you've started overfitting. Stop there. The --save-every flag means you have checkpoints to roll back to.

Code

Sane-defaults LoRA training command (annotated)·bash

python -m mlx_lm lora \
  --model mlx-community/Mistral-7B-Instruct-v0.3-4bit \
  --train \
  --data ./my-data \
  --fine-tune-type lora \
  --num-layers 16 \
  --batch-size 4 \
  --iters 200 \
  --learning-rate 5e-5 \
  --max-seq-length 2048 \
  --steps-per-report 10 \
  --steps-per-eval 25 \
  --save-every 100 \
  --adapter-path ./my-adapter

# Output during training:
#   Iter 10: Train loss 1.832, Learning Rate 5.000e-05, ...
#   Iter 25: Val loss 1.523, ...
#   Iter 100: Saving checkpoint ./my-adapter/0000100_adapters.safetensors
#   ...

Use a YAML config file for repeatable runs·yaml

# my-lora.yaml — one source of truth for a fine-tune run
model: "mlx-community/Mistral-7B-Instruct-v0.3-4bit"
train: true
data: "./my-data"
fine_tune_type: "lora"
num_layers: 16
batch_size: 4
iters: 200
learning_rate: 5e-5
max_seq_length: 2048
steps_per_report: 10
steps_per_eval: 25
save_every: 100
adapter_path: "./my-adapter"
seed: 42

# Then pass with -c:
#   python -m mlx_lm lora -c my-lora.yaml

Exercise

Take the YAML config above, adjust it for your dataset (set data, lower iters to 50 for a quick run, set max_seq_length to fit your longest example), and start a training run with python -m mlx_lm lora -c my-lora.yaml. Watch the train and val loss numbers in the report. Stop the run when val loss stops improving (or after the iters complete). Note the iteration count where val loss bottomed out — that's your effective training budget for this dataset.