Data Augmentation

The cheapest regularizer that exists

Data augmentation generates new training examples by applying label-preserving transformations to existing ones. For images: random crop, flip, color jitter, MixUp, CutMix, RandAugment. For text: synonym replacement, back-translation, dropout at the token level. For audio: pitch shift, time warp, noise injection.

Augmentation works because it forces the model to learn invariances ("the dog is still a dog if you crop it differently") rather than memorize specific pixel patterns. Stronger augmentation = stronger implicit regularization.

Tip: Aug pipelines compose cheaply. Start with the standard 'flip + crop + normalize' for vision, add ColorJitter for outdoor / natural imagery, add CutOut/MixUp for serious training. The default torchvision/timm pipelines are battle-tested.

Vision augmentation, in 2026

Strong baseline: RandomResizedCrop + RandomHorizontalFlip + ColorJitter + RandAugment + Normalize. Add MixUp or CutMix for an extra 1-2% on classification benchmarks.

Augmentation for text and audio

Text: SpecAugment for ASR, simple word dropout for many NLP tasks, back-translation for low-resource MT. Audio: SpecAugment, room impulse response convolution for speech robustness, SoX-style speed/pitch augmentation.

Principle: Augmentation is more important than the model architecture for most vision tasks. The single highest-leverage thing you can do for a stuck training run is upgrade the augmentation pipeline.

Code

Standard vision augmentation pipeline·python

from torchvision import transforms

train_tf = transforms.Compose([
    transforms.RandomResizedCrop(224),
    transforms.RandomHorizontalFlip(),
    transforms.ColorJitter(brightness=0.2, contrast=0.2, saturation=0.2),
    transforms.RandAugment(num_ops=2, magnitude=9),
    transforms.ToTensor(),
    transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]),
    transforms.RandomErasing(p=0.25),   # cutout
])

eval_tf = transforms.Compose([
    transforms.Resize(256),
    transforms.CenterCrop(224),
    transforms.ToTensor(),
    transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]),
])

The cheapest regularizer that exists

Vision augmentation, in 2026

Augmentation for text and audio

Code

External links

Exercise

Progress

Comments 0