Audio APIs

~22 min · audio, tts, transcription

Level 0Tokenizer

0 XP0/54 lessons0/10 achievements

0/120 XP to next level120 XP to go0% complete

OpenAI provides both speech-to-text (transcription) and text-to-speech (TTS) capabilities.

Text-to-Speech

The gpt-4o-mini-tts model supports expressive TTS with style instructions embedded in the input text. Available voices: alloy, echo, fable, onyx, nova, and shimmer.

Three audio surfaces, three jobs

(1) Text-to-speech (gpt-4o-mini-tts) — text in, audio file out. Use for voice replies you generate from chat output. (2) Speech-to-text (gpt-4o-transcribe) — audio file in, text out. Use for transcribing user audio uploads. (3) Realtime API — websocket, full-duplex audio in/out, low-latency. Use for live voice agents.

cwkPippa's TTS layer uses ElevenLabs (Joanne voice) for richer prosody than gpt-4o-mini-tts gives, with MD5-based caching to avoid regenerating identical lines. The OpenAI TTS surface is the right default when ElevenLabs-quality isn't needed.

Code

TTS with gpt-4o-mini-tts·python

# Classic Whisper
with open("recording.mp3", "rb") as audio_file:
    transcript = client.audio.transcriptions.create(
        model="whisper-1",
        file=audio_file,
        language="en",
        response_format="text",  # "text", "json", "srt", "vtt"
    )
print(transcript.text)

# GPT-4o powered (more accurate)
with open("recording.mp3", "rb") as audio_file:
    transcript = client.audio.transcriptions.create(
        model="gpt-4o-transcribe",
        file=audio_file,
    )

Transcription with gpt-4o-transcribe·python

response = client.audio.speech.create(
    model="tts-1",             # "tts-1", "tts-1-hd", "gpt-4o-mini-tts"
    voice="alloy",             # "alloy", "echo", "fable", "onyx", "nova", "shimmer"
    input="Hello! Welcome to our service.",
    response_format="mp3",     # "mp3", "opus", "aac", "flac", "wav", "pcm"
    speed=1.0,                 # 0.25 – 4.0
)

with open("speech.mp3", "wb") as f:
    f.write(response.content)

External links

Exercise

Build a 'speak this answer' flow: text → gpt-4o-mini-tts MP3 → save. Then reverse: an MP3 file → gpt-4o-transcribe → text. Roundtrip a paragraph and measure how much was preserved.