Raw Text Isn't Speakable Text

"Text is written for the eye. Speech is made for the ear. Normalization is the translation between them, and it's mostly judgment."

The Gap Between Written and Spoken

Hand a raw sentence to a synthesizer and the seams show immediately. $5 should become "five dollars," not "dollar sign five." 3pm is "three p.m." A trailing Dr. might be "Doctor" or "Drive" depending on context. Written language is full of shorthand the eye resolves instantly and the ear cannot. Normalization is the step that turns what's written into what should be said — the unglamorous work that decides whether the voice sounds fluent or robotic.

It's Preservation, Not Just Expansion

The naive view is "expand everything." But normalization is judgment in both directions: some symbols should be spoken out, and some should be preserved because they carry meaning the voice needs. A technical symbol in code or math shouldn't be mangled into prose. Getting this right is why normalization is a real subsystem with versions and tests, not a pile of find-and-replace. Expand what helps; keep what matters.

Deterministic, Versioned, and First

Two properties make normalization safe to build a cache on. It's deterministic: the same text under the same rules produces the same output, every time. And it's versioned: the dictionary and rule set have a version that rides in the cache identity, so changing how something is spoken changes the key and correctly misses the old cache. That's why normalization runs first, before the identity is computed — the cache is keyed on the sound the engine will actually make.

Code

Normalization: expand what helps, keep what matters·python

def normalize(text: str, s: Settings) -> str:
    # Raw text is for the eye; normalized text is for the voice.
    text = expand_numbers_and_units(text)     # '$5'  -> 'five dollars'
    text = expand_abbreviations(text)         # 'Dr.' -> 'Doctor' (in context)
    text = apply_dictionary(text, s)          # per-term pronunciations
    text = preserve_technical_symbols(text)   # keep symbols that carry meaning
    return text

# DETERMINISTIC + VERSIONED: same text + same dict version -> same output
# -> same cache key. Change the rules and the version (and the key) change too.

Exercise

Write a raw sentence packed with things the eye resolves silently — a price, a time, an abbreviation, a URL, a technical symbol. Under it, write how it should actually sound. Mark every token where the right reading depends on context (is 'St.' Saint or Street?). Those context-dependent tokens are exactly where normalization stops being replacement and becomes judgment.

Hint

The easy tokens have one spoken form. The interesting ones have two, and only context picks. Count how many of your tokens are ambiguous — that count is the size of the judgment problem a real normalizer has to solve.