Experiment Tracking

Why you need a tracker

Within a week of any serious project, you have run a hundred experiments with different hyperparameters and you have lost track of which one produced the model on disk. An experiment tracker fixes that: every run is logged with code commit, config, metrics, and artifacts. Run comparisons become a UI question instead of a forensic exercise.

Three popular trackers

Weights & Biases (W&B) — managed cloud, gorgeous UI, free for individuals, paid for teams. The de facto standard in industry.
MLflow — open source, self-host or cloud-host, integrates with Databricks. Less polished UI, more flexible deployment.
TensorBoard — PyTorch native, runs locally, simple, no extra service to set up. Great for solo work.

Tip: Pick one tracker per team and stick with it. The cost of switching is real (every previous run lives in the old tool). For solo work, TensorBoard is fine; for teams, W&B usually wins.

What to log

Config — every hyperparameter, the git commit hash, the dataset version.
Metrics — loss (train/val), task metric (accuracy/F1/etc), gradient norm, learning rate, throughput.
Artifacts — final checkpoint, best checkpoint, sample predictions, confusion matrices.
System — GPU utilization, memory, training duration.

Principle: If you can't reproduce an experiment from the tracker entry alone, the tracker is decoration not infrastructure. Make sure the config + commit hash are enough to rerun.

Code

W&B in five lines·python

import wandb

wandb.init(project="dl-foundations", config=cfg.__dict__)
for epoch in range(cfg.epochs):
    train_loss = train_one_epoch(model, train_loader, opt, device)
    val_acc = evaluate(model, val_loader, device)
    wandb.log({"train_loss": train_loss, "val_acc": val_acc, "epoch": epoch,
               "lr": opt.param_groups[0]["lr"]})
wandb.finish()

TensorBoard for local-only logging·python

from torch.utils.tensorboard import SummaryWriter

writer = SummaryWriter(log_dir="runs/exp001")
for step, (xb, yb) in enumerate(train_loader):
    loss = train_step(model, xb, yb, opt)
    writer.add_scalar("train/loss", loss, step)
    if step % 100 == 0:
        writer.add_scalar("lr", opt.param_groups[0]["lr"], step)
writer.close()
# tensorboard --logdir runs

Why you need a tracker

Three popular trackers

What to log

Code

External links

Exercise

Progress

Comments 0