Your Brain Is a Lossy Codec

You Don't See What's There. You See a Sample.

The light hitting your retina spans a continuous spectrum of wavelengths. Your eyes have three types of cone cells. Three. Not three thousand. Just enough to give you red, green, blue — and from those three samples, your brain reconstructs every color you've ever seen. Birds have four cones. Mantis shrimp have sixteen. They literally sample reality at higher resolution than you do.

Your ears do the same trick. ~20 Hz to 20,000 Hz of continuous pressure waves get sampled into a discrete neural code. Frequencies above 20 kHz exist — bats hear them — you don't. Your hardware decided sampling rate.

Memory is the most aggressive lossy codec of all. You don't remember last Thursday as a continuous 24-hour video. You remember keyframes — a few smells, two emotional spikes, one sentence somebody said. The rest is reconstructed on demand by a generative model that lives between your ears, sometimes badly.

Why Courts Distrust Eyewitnesses

If your brain were a perfect recorder, an eyewitness would be the gold standard of evidence. But brains aren't recorders — they're lossy codecs with creative reconstruction. So courts treat eyewitness testimony with measured skepticism. The witness isn't lying; their codec is doing what codecs do.

This isn't a bug. It's what makes you fast. A perfect memory of every sensory input would crush you under data. Sampling and lossy compression are how you survive the firehose of reality.

You are running lossy compression on the universe at all times. AI just borrows your tricks — coarsely and explicitly.

The Engineering Translation

Audio: 44,100 Hz sample rate, 16-bit depth — Nyquist's theorem says that's enough to perfectly reconstruct frequencies up to 22 kHz, which is past your hearing limit. Video: 30 frames per second — exploits your visual system's sample-and-blend behavior. Image compression (JPEG): throws away high-frequency detail your eyes don't notice. Neural network embeddings: project a near-infinite meaning space into a 4,096-dim sample.

Same idea, different domains. The art is choosing what to keep.

Code

Sampling a continuous signal·python

import numpy as np

# Continuous signal — a sine wave at 5 Hz
t_continuous = np.linspace(0, 1, 10_000)         # very fine
signal_continuous = np.sin(2 * np.pi * 5 * t_continuous)

# Sampling at 50 Hz — way above Nyquist (need >= 10 Hz)
t_sampled = np.linspace(0, 1, 50)
signal_sampled = np.sin(2 * np.pi * 5 * t_sampled)

# Reconstruction is faithful enough for AI work
print(signal_sampled.shape)   # (50,) — discrete points
# Plot both with matplotlib to see how 50 samples preserves a 5 Hz wave.

Exercise

Pick a single memory from this week. Write down 3 sensory details you remember vividly. Then write down 3 details you reconstructed (had to guess based on context). Notice the ratio. That ratio is your sample rate vs. your reconstruction.

Hint

If you can't easily separate the two, that's the lossy codec doing its job — your brain merges samples and reconstructions seamlessly. That's a feature.