End-to-End Walkthrough — Fine-Tune, Merge, Serve

The whole pipeline in one place

Five lessons of pieces; one lesson putting them together. We're going to fine-tune a 7B instruct base on a tiny domain dataset, merge the adapter into a deployable model, serve it through mlx-lm's HTTP server, and call it from openai-python. End to end on a single 32 GB Mac, no GPU rental required.

The full sequence

Pick a base. mlx-community/Mistral-7B-Instruct-v0.3-4bit — 7B Q4, fits in ~5 GB of unified memory, decent instruction following baseline.
Prepare data. Build a small train.jsonl + valid.jsonl in the chat format (lesson 2). 100–300 examples is plenty for the demo.
Train. Run mlx_lm lora --train with sane defaults from lesson 3. Watch val loss; stop when it plateaus.
Merge. Run mlx_lm fuse to fold the adapter into the base, producing a new model directory.
Serve. Start mlx_lm server --model ./my-fused in one terminal.
Use. Hit the server from openai-python in another terminal — the fine-tuned model now answers in your domain's voice.

What this exercise should leave you with

Three things. First, a working artifact you can refer back to — a file layout, a YAML config, a training command, a fuse command, a server command. Second, the muscle memory that this whole pipeline is one person on one Mac in an afternoon, not a team-and-cloud project. Third, a baseline you can iterate on — your next fine-tune is a variation of this walkthrough, not a rebuild from scratch.

The artifacts the walkthrough produces

./my-data/{train,valid}.jsonl — your curated dataset.
./my-lora.yaml — the training config (one source of truth).
./my-adapter/ — the trained LoRA adapter (small file, the actual learned correction).
./my-fused/ — the merged deployable model directory.
An HTTP endpoint at http://localhost:8080/v1 serving your fine-tune to anything OpenAI-compatible.

Where to go from here

Push the fused model to your own mlx-community-style HF repo with --upload-repo if you want others to pull it. Or keep iterating — train a second adapter on a different slice of your domain and compare them at inference. Or jump to prod.lesson1 to wrap your fine-tune in a real FastAPI service with worker management, when the built-in server hits its limits.

Code

Step 1-2 — Pick the base, prepare a tiny dataset·bash

mkdir -p my-data
# Hand-write at least 50 chat-format examples in train.jsonl
# and ~10 in valid.jsonl (lesson 2 has the format).
# You can also use a public dataset like mlx-community/wikisql for the demo.

Step 3 — Train the LoRA adapter·bash

# my-lora.yaml from lesson 3 contains all the flags as one source of truth.
python -m mlx_lm lora -c my-lora.yaml

# Or inline:
python -m mlx_lm lora \
  --model mlx-community/Mistral-7B-Instruct-v0.3-4bit \
  --train --data ./my-data \
  --fine-tune-type lora --num-layers 16 \
  --batch-size 4 --iters 200 --learning-rate 5e-5 \
  --max-seq-length 2048 \
  --steps-per-eval 25 --save-every 100 \
  --adapter-path ./my-adapter

Step 4 — Fuse the adapter; Step 5 — Serve·bash

# Fuse
python -m mlx_lm fuse \
  --model mlx-community/Mistral-7B-Instruct-v0.3-4bit \
  --adapter-path ./my-adapter \
  --save-path ./my-fused

# Serve (in one terminal — keeps running)
python -m mlx_lm server --model ./my-fused

Step 6 — Hit the fine-tune from a client·python

# In a separate terminal:
from openai import OpenAI

client = OpenAI(base_url="http://localhost:8080/v1", api_key="x")

resp = client.chat.completions.create(
    model="./my-fused",
    messages=[
        {"role": "system", "content": "You are an expert in <your domain>."},
        {"role": "user",   "content": "<a question your fine-tune should answer well>"},
    ],
    max_tokens=120,
    temperature=0.7,
)
print(resp.choices[0].message.content)

Exercise

Run the entire walkthrough end-to-end on a domain you care about. Spend 70% of your time on the data (lesson 2), 20% running the training and watching loss, 10% on the fuse + serve + test loop. Save the YAML config so you can re-run the whole thing with one command later. The exercise is to feel that the pipeline is repeatable — your second domain fine-tune should take half the time of the first.