SFTTrainer: The High-Level API

~24 min · trl, sft-trainer, sft-config, supervised-fine-tuning

Level 0Observer

0 XP0/43 lessons0/11 achievements

0/120 XP to next level120 XP to go0% complete

The recommended path

TRL's SFTTrainer is the recommended way to do supervised fine-tuning. It handles tokenization, formatting, loss masking, chat templates, and the training loop in one clean API. You provide the model, the LoRA config, the dataset, and an SFTConfig — the trainer does the rest.

Key SFTConfig parameters

Parameter	What it controls	Typical value
`max_seq_length`	Max sequence length for training.	1024–4096
`packing`	Pack multiple short examples into one sequence.	`True` — saves training time
`dataset_text_field`	Which field contains the text.	`None` for auto-detect on chat format
`gradient_checkpointing`	Recompute activations instead of storing.	`True` for large models
`bf16`	Use bfloat16 mixed precision.	`True` on Ampere+ GPUs

Code

End-to-end SFTTrainer setup·python

from trl import SFTConfig, SFTTrainer
from peft import LoraConfig
from datasets import load_dataset

# Load dataset (many formats supported)
dataset = load_dataset("json", data_files="training_data.jsonl", split="train")
eval_dataset = load_dataset("json", data_files="val.jsonl", split="train")

# LoRA config
peft_config = LoraConfig(
    r=16, lora_alpha=32,
    target_modules="all-linear",
    lora_dropout=0.05, bias="none",
    task_type="CAUSAL_LM",
)

# Training config (TRL v1.0: SFTConfig extends TrainingArguments)
training_args = SFTConfig(
    output_dir="./results",
    num_train_epochs=3,
    per_device_train_batch_size=4,
    gradient_accumulation_steps=4,    # effective batch = 4 * 4 = 16
    learning_rate=2e-4,
    lr_scheduler_type="cosine",
    warmup_ratio=0.1,
    logging_steps=10,
    save_strategy="epoch",
    eval_strategy="epoch",
    bf16=True,
    gradient_checkpointing=True,
    max_seq_length=2048,
    packing=True,
    dataset_text_field=None,           # auto-detect 'messages'
    dataset_kwargs={"assistant_only_loss": True},  # train only on assistant tokens
)

trainer = SFTTrainer(
    model=model,
    args=training_args,
    train_dataset=dataset,
    eval_dataset=eval_dataset,
    processing_class=tokenizer,
    peft_config=peft_config,
)

trainer.train()
trainer.save_model("./final-adapter")

External links

Exercise

Set up SFTTrainer with the config above on a 200-example dataset. Run for 3 epochs. Watch the eval loss in W&B or the console. Compare your trained adapter's outputs to the base model's on 5 held-out test prompts. Document the qualitative difference.