Practical: Fine-Tuning for Classification

Everything in the track, in one workflow

This is the capstone — the whole track collapsed into a single runnable recipe. Read the Code section as a checklist of what you've learned: a preset (lesson 2) gives you a pretrained backbone and its matched tokenizer (lesson 3); num_classes shapes the classification head (lesson 4); enable_lora makes the fine-tune cheap (lesson 6); and save writes a single .keras file you can ship. Nothing here is new — it's the assembly.

The details that decide success

A few choices in this recipe are worth defending. AdamW over plain Adam: the decoupled weight decay is the standard for fine-tuning transformers and reliably generalizes a little better. LoRA on the capstone: even for a model BERT's size it's worth turning on, because it makes the run repeatable on modest hardware and keeps the saved artifact small. A validation set is non-negotiable — without validation_data you can't see overfitting coming, and three epochs on a pretrained model can overfit a small dataset fast. The single most common failure here isn't the model at all: it's an unbalanced or mislabeled dataset. Garbage labels in, confident garbage out.

From here, it's all variations

Once this runs end-to-end, the door is open. Swap the dataset and you have a customer-support router; swap the backbone for a multilingual preset and it works on non-English text; swap num_classes and the same code does topic tagging or content moderation. The capstone isn't a finish line — it's the template every later NLP feature reuses.

Code

End-to-end: load, LoRA, compile, fit, save a fine-tuned classifier·python

import keras
import keras_hub

# Load pretrained classifier
classifier = keras_hub.models.BertClassifier.from_preset(
    "bert_base_en",
    num_classes=3,
)

# Enable LoRA for efficient fine-tuning
classifier.backbone.enable_lora(rank=4)

# Compile
classifier.compile(
    optimizer=keras.optimizers.AdamW(5e-5),
    loss="sparse_categorical_crossentropy",
    metrics=["accuracy"],
)

# Train
classifier.fit(
    train_ds,
    validation_data=val_ds,
    epochs=3,
)

# Save the fine-tuned model
classifier.save("my_classifier.keras")

Practical: Fine-Tuning for Classification

Everything in the track, in one workflow

The details that decide success

From here, it's all variations

Code

External links

Exercise

Progress

Comments 0