C.W.K.
Stream
Lesson 01 of 05 · published

Why Revisit Convolutions?

~12 min · convolution, fft, shift-invariance

Level 0Observer
0 XP0/50 lessons0/14 achievements
0/100 XP to next level100 XP to go0% complete

The case for going back to convolutions

Convolutions were the dominant sequence-mixing primitive before attention won. They have well-understood mathematical properties, well-tested kernels, and three structural advantages worth re-examining: shift-invariance (the same pattern is recognized regardless of where it appears in the sequence), natural FFT computation at O(L log L) cost, and parameter sharing across positions (a kernel of size k is one set of weights applied everywhere).

The historical objection: standard 1D convolutions have a fixed receptive field. A convolution with kernel size k can only mix information within a k-token window. To handle long-range dependencies you'd need astronomically large kernels — and the kernel weights would have to be stored, requiring proportional memory.

The implicit-filter unlock

Hyena's insight was: you don't have to store the filter. Generate it on-the-fly from a small neural network that takes position as input and outputs filter weights. With this trick, the filter can be arbitrarily long — say, the full sequence length — without ever existing in memory all at once. The FFN that generates the filter has a fixed parameter count regardless of how long the filter is.

This is the same engineering pattern as positional encodings: instead of learning explicit position embeddings (one parameter per position), use a function (RoPE rotates by an angle that's a function of position). Implicit filters are the convolution analog. The compute cost shifts from "store a giant filter" to "call a small FFN at every position" — which on modern hardware is a much better trade.

External links

Exercise

Implement a tiny implicit-filter convolution: a 3-layer MLP that takes (positional encoding, layer index) and outputs a 1024-tap filter, applied to a 1024-length signal via torch.fft.rfft / irfft. Verify the FFT-based application gives the same result as an explicit conv1d with the same filter. Time both. The FFT path will pull ahead at ~512+ filter taps — that crossover is where Hyena's story starts.

Progress

Progress is local-only — sign in to sync across devices.
Spotted a bug or have feedback on this page?Report an Issue

Comments 0

🔔 Reply notifications (sign in)
Sign inPlease sign in to comment.

No comments yet — be the first.