Evaluation Mode

Train mode and eval mode are different functions

Some PyTorch layers — Dropout, BatchNorm, LayerNorm with affine=True usually behaves the same — change behavior depending on whether the model is in training or evaluation mode. Dropout zeros random units in train, identity in eval. BatchNorm uses batch statistics in train and running statistics in eval. Forgetting to switch causes silent metric noise.

Switch with model.train() and model.eval(). They are recursive — calling on the top-level module switches every submodule.

Tip: Make a habit: every training loop starts with model.train(), every validation loop starts with model.eval(). Even if you 'know' you set it correctly five lines ago.

Pair eval mode with no_grad/inference_mode

For evaluation you don't need gradients, and computing them wastes memory and time. Use torch.inference_mode() (newer, faster) or torch.no_grad() (older, equivalent for most use cases).

Common eval bugs

Computing accuracy on the GPU then printing CPU values — every .item() forces a CUDA sync. Aggregate on the GPU, sync once at the end of the epoch.

Mismatched preprocessing — training applies augmentations (random crop, flip, color jitter), evaluation should apply only deterministic preprocessing (resize, center crop, normalize). Mixing the two is one of the most common silent bugs in vision training.

Principle: Eval is not training. Same data layer, different mode, different preprocessing, no gradient graph. Get this distinction in your bones early — it saves countless 'why is my val loss higher than my train loss' debugging sessions.

Code

A clean evaluation function·python

import torch

@torch.inference_mode()
def evaluate(model, loader, device):
    model.eval()
    correct, total, total_loss = 0, 0, 0.0
    loss_fn = torch.nn.CrossEntropyLoss(reduction="sum")
    for xb, yb in loader:
        xb, yb = xb.to(device, non_blocking=True), yb.to(device, non_blocking=True)
        logits = model(xb)
        total_loss += loss_fn(logits, yb).item()
        preds = logits.argmax(dim=-1)
        correct += (preds == yb).sum().item()
        total   += yb.size(0)
    return correct / total, total_loss / total

Train mode and eval mode are different functions

Pair eval mode with no_grad/inference_mode

Common eval bugs

Code

External links

Exercise

Progress

Comments 0