The Ultimate Normalizer: Compressing Extreme Scales

When Numbers Span Universes

Some quantities live across wild scales. Brightness of stars from the Sun to a distant galaxy. Earthquake magnitudes from a barely-felt tremor to a continent-shifting catastrophe. Sound from a whisper to a jet engine. Probabilities in a language model from "very common" to "essentially never."

Plotting these on a linear scale fails — the small values get crushed into invisibility. Logarithms compress. They turn exponential ranges into linear-feeling ones.

Decibels: Your Ears Already Do This

The decibel scale measures sound intensity logarithmically: $dB = 10 lo g_{10} (I / I_{0})$ . 0 dB is the threshold of hearing. 60 dB is conversation. 120 dB is a jet engine. Each 10 dB step is a 10× change in actual sound energy. Your ears already perceive sound this way — you hear "twice as loud" when intensity multiplies by ~10. Engineers built dB to match human perception, which itself is logarithmic.

Why AI Loves Logs

Three jobs:

Numerical stability. Multiplying tiny probabilities together ( $1 0^{- 50} \times 1 0^{- 50}$ ) underflows to zero in float32. Adding their logs (-50 + -50 = -100) doesn't.
Loss functions. Cross-entropy loss is $- \sum y_{i} lo g (\overset{y}{^}_{i})$ . The log gives gentler gradients near correct predictions and harsher penalties for very-wrong predictions — exactly the asymmetry we want.
Visualization. Loss curves are usually plotted on a log y-axis so the early dramatic drops don't compress the late subtle improvements out of view.

Log = compressor for extreme ranges. Linear thinking dies on data spanning orders of magnitude. Switch to log and the structure reappears.

Code

Compress wild ranges to linear-feeling steps·python

import numpy as np

# Star brightness — orders of magnitude apart
brightness = np.array([1e-3, 1e-1, 1e1, 1e3, 1e5, 1e7])

# Linear plot would compress small ones
print(brightness)

# Log scale makes them comparable
print(np.log10(brightness))   # [-3 -1  1  3  5  7]

# Decibels — exact same idea on sound intensity
intensity = np.array([1e-12, 1e-9, 1e-6, 1e-3, 1])
dB = 10 * np.log10(intensity / 1e-12)
print(dB)       # [  0  30  60  90 120]

Exercise

Generate 100 random numbers between 1 and 10000. Plot a histogram on a linear x-axis, then on a log x-axis. Notice how the log-scale histogram becomes interpretable while the linear one is dominated by the few large values.

Hint

np.random.uniform(1, 10000, 100) then plt.xscale('log'). Many real-world quantities (file sizes, incomes, city populations) need log scales to be readable.

The Ultimate Normalizer: Compressing Extreme Scales

When Numbers Span Universes

Decibels: Your Ears Already Do This

Why AI Loves Logs

Code

External links

Exercise

Progress

Comments 2

1~10000에 걸쳐 자릿수가 고르게 퍼지도록 로그 균등 생성 (현실 데이터의 전형)

선형 x축 히스토그램

로그 x축 히스토그램 — 로그 간격 구간 사용