Quiz · 4 questions
⚡ Why Transformers
Parallel sequence modeling and the bottleneck it broke
Level 0Token
0 XP0/94 lessons0/10 achievements
0/120 XP to next level120 XP to go0% complete
Quiz
01What is the primary architectural bottleneck of RNNs that the Transformer eliminates?
Hint
Think about what the GPU is good at and what the RNN forces it to wait on.
02What is the asymptotic time/memory complexity of standard self-attention in sequence length n?
Hint
Look at the shape of Q @ K^T.
03Which architecture shape is used by GPT-5, LLaMA 4, and Claude 3.7 Sonnet?
Hint
What attention pattern do you need if you generate one token at a time?
04What does Chinchilla's compute-optimal rule suggest for training a Transformer?
Hint
Look up Chinchilla's rule and remember the number 20 — but also why modern training doesn't follow it.
Comments 0
🔔 Reply notifications (sign in)Sign in — Please sign in to comment.
No comments yet — be the first.