Convolutions: Shared Filters

What a convolution actually does

A 2-D convolution slides a small filter (e.g. 3×3 weights) across an input image and at each position computes a weighted sum of the local pixel patch. The same filter is reused at every position — that's the parameter sharing that makes CNNs efficient. A single 3×3 conv with 32 input channels and 64 output channels has 3*3*32*64 + 64 = 18496 parameters, regardless of image size.

You stack many filters per layer (the channel dimension) and many layers (the depth), with non-linearities between them. Early layers learn edge detectors and color blobs; mid layers learn textures and shape parts; deep layers learn object-shaped features.

Tip: Read every nn.Conv2d as: 'apply N learned filters of size kxk across the spatial dimensions, with stride s and padding p, producing an output of shape [B, N, H_out, W_out].' The exact output shape comes from floor((H + 2p - k)/s) + 1.

The hyperparameters that matter

kernel_size — usually 3, sometimes 1 (channel-mix), sometimes 7 (early layers).
stride — 1 (same spatial size) or 2 (downsample).
padding — usually k//2 for 'same' padding (output same size as input when stride=1).
in_channels / out_channels — width of the network at this layer.
groups — 1 (standard) or in_channels (depthwise, for efficient mobile architectures).

Why convolutions generalize

Two reasons. Translation equivariance: if you shift the input, the output shifts the same way — the model doesn't need to relearn features for every position. Parameter sharing: each filter is reused across the image, so you have far fewer parameters and they're each trained on far more data than they would be in a fully-connected layer.

Principle: Convolutions encode locality and translation as built-in assumptions. When those assumptions match the data (images, audio spectrograms), CNNs are dramatically more parameter-efficient than fully-connected models. When they don't (tabular, set-valued data), use a different architecture.

Code

Hand-trace a Conv2d output shape·python

import torch
import torch.nn as nn

x = torch.randn(8, 3, 224, 224)              # [B, C, H, W]

conv = nn.Conv2d(in_channels=3, out_channels=64,
                 kernel_size=3, stride=2, padding=1)
y = conv(x)
print(y.shape)  # torch.Size([8, 64, 112, 112])
# H_out = floor((224 + 2*1 - 3) / 2) + 1 = 112

# Channel-mix 1x1 conv (no spatial change, just learns linear combos of channels)
conv1x1 = nn.Conv2d(64, 32, kernel_size=1)
print(conv1x1(y).shape)  # torch.Size([8, 32, 112, 112])

Depthwise separable conv (mobile efficiency trick)·python

class DepthwiseSeparable(nn.Module):
    def __init__(self, in_ch, out_ch):
        super().__init__()
        self.dw = nn.Conv2d(in_ch, in_ch, 3, padding=1, groups=in_ch, bias=False)
        self.pw = nn.Conv2d(in_ch, out_ch, 1, bias=False)
    def forward(self, x):
        return self.pw(self.dw(x))
# Far fewer params than a full Conv2d(in_ch, out_ch, 3) — same receptive field

What a convolution actually does

The hyperparameters that matter

Why convolutions generalize

Code

External links

Exercise

Progress

Comments 0