Classic CNN Lineage

Five architectures, fifteen years, one through-line

LeNet-5 (1998) — Yann LeCun's network for digit recognition. Two convolutions, two fully-connected layers, sigmoid activations. Worked, but the era's computers couldn't scale it.

AlexNet (2012) — five convolutions, three fully-connected layers, ReLU activations, dropout, GPU training. Won ImageNet by a huge margin and lit the deep-learning fuse.

VGG (2014) — depth via stacks of small (3×3) convolutions. Simple, regular, easy to read. Heavy on parameters; light on inductive bias beyond "deeper is better".

GoogLeNet / Inception (2014) — multi-scale within a layer (1×1, 3×3, 5×5 in parallel). Introduced 1×1 convolutions for cheap channel mixing.

ResNet (2015) — residual connections that made 100+ layer networks trainable. Still a common backbone in 2026 because the design ideas held up.

EfficientNet (2019), ConvNeXt (2022) — careful scaling of width/depth/resolution; design choices ported from transformer practice (LayerNorm, GELU, larger kernels).

Tip: If you read one CNN paper end-to-end, make it ResNet. The residual connection is the most copied design idea of the last decade and you'll see it in every modern architecture.

Why this matters in 2026

You probably won't train a CNN from scratch — you'll use a pretrained backbone from torchvision or timm. But knowing the lineage tells you which backbone to pick: ResNet50 for general use, ConvNeXt-Tiny for modern strong baselines, EfficientNet for mobile/edge, ViT-B/16 if you want a transformer.

Principle: CNN architecture decisions in 2026 are mostly 'pick the right pretrained backbone for your task and fine-tune'. Knowing why each backbone was designed the way it was makes that pick competent rather than random.

Code

Pretrained backbones via torchvision·python

import torch.nn as nn
import torchvision.models as tvm
from torchvision.models import (
    ResNet50_Weights, ConvNeXt_Tiny_Weights, EfficientNet_B0_Weights,
)

resnet = tvm.resnet50(weights=ResNet50_Weights.IMAGENET1K_V2)
convnext = tvm.convnext_tiny(weights=ConvNeXt_Tiny_Weights.IMAGENET1K_V1)
effnet = tvm.efficientnet_b0(weights=EfficientNet_B0_Weights.IMAGENET1K_V1)

# Replace the head for your N-class task
def replace_head(model, n_classes):
    if hasattr(model, "fc"):
        model.fc = nn.Linear(model.fc.in_features, n_classes)
    elif hasattr(model, "classifier"):
        model.classifier[-1] = nn.Linear(model.classifier[-1].in_features, n_classes)
    return model

Five architectures, fifteen years, one through-line

Why this matters in 2026

Code

External links

Exercise

Progress

Comments 0