Mental model: Think of voice generation as having three levels of control, like a sound studio mixing board. The first channel is what is said (the text). The second is who says it (the voice identity). The third is how they say it (emotion, pacing, emphasis). Each channel is independently adjustable, and the magic happens when you dial all three precisely.
Channel 1: What — Text-to-Speech (TTS)
At its most basic, TTS converts written text into spoken audio. Modern TTS has moved far beyond the robotic voices of GPS navigation. Current models produce speech that's nearly indistinguishable from human recordings in blind tests.
Key considerations:
- Pronunciation: Most models handle standard words well, but technical terms, brand names, and unusual words may be mispronounced. Many platforms allow phonetic hints.
- Punctuation matters: Commas create pauses. Periods create stops. Em-dashes create dramatic pauses. Question marks shift intonation upward. Your punctuation is a delivery script.
- Length and pacing: Short sentences sound punchy and urgent. Long flowing sentences sound contemplative. The structure of your text directly shapes the vocal performance.
Channel 2: Who — Voice Identity and Cloning
Voice cloning creates a synthetic version of a specific voice from a sample recording:
- Instant cloning: 15–60 seconds of sample audio → usable clone. Quick but less accurate.
- Professional cloning: 5–30 minutes of studio recording → high-fidelity clone. Used for commercial voice actors licensing their voice.
ElevenLabs reports 94% voice similarity in cloning, meaning listeners can reliably identify the cloned voice as the same speaker. This enables consistent character voices across long projects (audiobooks, series, games) without scheduling the original speaker for every session.
Channel 3: How — Emotional Delivery and Style Direction
The most nuanced and challenging channel. Telling a voice model to "sound excited" or "speak sadly" requires the model to adjust multiple vocal parameters simultaneously:
- Prosody: The rhythm, stress, and intonation of speech. Happy speech has wider pitch variation and faster tempo. Sad speech is flatter and slower.
- Emphasis: Which words are stressed. "I didn't say HE stole it" vs. "I didn't say he STOLE it" — same words, different meanings.
- Pacing: Speed of delivery, placement of pauses, breath patterns.
- Vocal quality: Breathy, gravelly, clear, strained, whispered, shouted.
- Voice generation has three control channels: what (text), who (voice identity), how (emotional delivery).
- Voice cloning achieves ~94% similarity from small samples — powerful but demands ethical responsibility.
- Emotional delivery is controlled through style parameters AND through the text structure itself.
- Punctuation and sentence structure are your primary "direction" tools — write how you want it spoken.