Quiz · 4 questions
🔤 Tokenization
How text becomes integers
Level 0Token
0 XP0/94 lessons0/10 achievements
0/120 XP to next level120 XP to go0% complete
Quiz
01Which subword tokenization algorithm does GPT-4 use?
Hint
It's the same algorithm as GPT-2 and GPT-3, just with a larger vocabulary.
02Why can a model fail at counting characters in 'strawberry'?
Hint
What does the model literally see when you type 'strawberry'?
03What approximate vocabulary size does Gemma 3 use?
Hint
It's the largest vocabulary among major open-weight models — designed for many languages.
04Why are output tokens typically 4-10× more expensive than input tokens?
Hint
Think about prefill vs decode — one is a big matmul, the other is many small ones.
Comments 0
🔔 Reply notifications (sign in)Sign in — Please sign in to comment.
No comments yet — be the first.