Text Classification

The 80% NLP task

Text classification — sentiment, topic, intent, spam — is the single most common thing teams actually ship with NLP. And it's where transfer learning earns its keep: instead of training a language model from scratch, you take a backbone that already understands English and bolt a small classification head on top. BertClassifier.from_preset(..., num_classes=2) does exactly that in one line — pretrained backbone, fresh head sized to your label count.

The three-line rhythm

Every Keras training loop is the same three beats, and the Code section shows them: load, compile, fit. The details worth internalizing live in compile. The learning rate is tiny — 5e-5, not the 1e-3 you'd use training from scratch — because you're nudging an already-smart model, not teaching it from zero; too high a rate erases the pretraining. The loss is sparse_categorical_crossentropy because labels are integers (0, 1, 2…), not one-hot vectors — pick the wrong one and the shapes won't match. Three epochs is usually plenty: a pretrained model converges fast, and more epochs mostly buys you overfitting.

Code

Load, compile, and fit a BERT sentiment classifier·python

import keras
import keras_hub

# Load BERT for classification
classifier = keras_hub.models.BertClassifier.from_preset(
    "bert_base_en",
    num_classes=2,  # Binary sentiment
)

# Compile and train
classifier.compile(
    optimizer=keras.optimizers.Adam(5e-5),
    loss="sparse_categorical_crossentropy",
    metrics=["accuracy"],
)
classifier.fit(train_ds, validation_data=val_ds, epochs=3)

The 80% NLP task

The three-line rhythm

Code

External links

Exercise

Progress

Comments 0