What Fine-Tuning Actually Does

Same training loop, different data

Under the hood, fine-tuning is gradient-based optimization — exactly the same process used to pre-train the model — applied to your data for a much smaller number of steps.

Forward pass: your example flows through the model, producing a prediction (next-token distribution).
Loss: the prediction is compared to the desired output. Cross-entropy is the standard.
Backward pass: gradients are computed — how much each parameter contributed to the error.
Update: the optimizer (typically AdamW) nudges each parameter slightly to reduce the loss.
Repeat: across hundreds or thousands of examples, for a few epochs.

Transfer learning is the magic

The pre-trained model already knows language, basic reasoning, and a vast amount of world knowledge. Fine-tuning transfers that knowledge into your specific domain. You are not teaching it English — you are teaching it your English.

This is why fine-tuning works with hundreds-to-thousands of examples instead of trillions. The foundation already exists; you are adjusting its expression. Cleanest analogy: pre-training is medical school, fine-tuning is the cardiology fellowship. You do not re-learn anatomy.

Where the changes live

Full fine-tuning updates every parameter. PEFT methods (LoRA, QLoRA, DoRA — Track 4) update only a small added subset, typically 0.1–1% of parameters, which both reduces compute cost and protects the base model's general abilities from being overwritten.

Code

What a single training step looks like in PyTorch·python

import torch
from torch.optim import AdamW

model.train()
optimizer = AdamW(model.parameters(), lr=2e-5)

for batch in train_loader:
    # 1. Forward pass — model produces logits over the vocab
    outputs = model(**batch, labels=batch["input_ids"])
    loss = outputs.loss

    # 2. Backward pass — compute gradients
    loss.backward()

    # 3. Update — optimizer nudges weights using the gradients
    optimizer.step()

    # 4. Reset gradients for the next batch
    optimizer.zero_grad()

    print(f"loss={loss.item():.4f}")
# That's the entire core. Everything else (LoRA, mixed precision,
# gradient checkpointing) is optimization wrapped around these four lines.

What Fine-Tuning Actually Does

Same training loop, different data

Transfer learning is the magic

Where the changes live

Code

External links

Exercise

Progress

Comments 0