Feature Extraction

Step 1: freeze the base, train only the new head

Feature extraction is the safest, fastest form of transfer learning. You set base_model.trainable = False so every pretrained weight is locked, then bolt a tiny trainable head on top. Only the head learns — which means most of the network never computes gradients, training is cheap, and there is no way to damage the pretrained features. This is always your first move; fine-tuning (next lesson) only earns its complexity if this baseline plateaus.

Two details that trip people up

The head is deliberately small: a GlobalAveragePooling2D collapses the backbone's feature map into one vector per image, a Dropout guards against overfitting on your small dataset, and a single Dense sized to your classes does the prediction. Two non-obvious things in the Code section earn their keep:

training=False when you call the base model — this keeps BatchNorm layers in inference mode so their running statistics aren't disturbed by your data while the weights are frozen.
Using the functional keras.Input form rather than wrapping in Sequential — it makes the frozen-base / trainable-head boundary explicit and is what the fine-tuning lesson builds on.

Code

Frozen backbone + custom head (feature extraction)·python

# Freeze the pretrained weights
base_model.trainable = False

# Add a custom classification head
inputs = keras.Input(shape=(224, 224, 3))
x = base_model(inputs, training=False)  # Keep BN in inference mode
x = layers.GlobalAveragePooling2D()(x)
x = layers.Dropout(0.2)(x)
outputs = layers.Dense(5, activation="softmax")(x)

model = keras.Model(inputs, outputs)
model.compile(
    optimizer="adam",
    loss="categorical_crossentropy",
    metrics=["accuracy"],
)

# Train only the head (~5-10 epochs)
model.fit(train_ds, epochs=10, validation_data=val_ds)

Step 1: freeze the base, train only the new head

Two details that trip people up

Code

External links

Exercise

Progress

Comments 0