Quiz · 5 questions
🧱 Dense Models — The Default Frontier
Every token activates every parameter — the simplest thing that works
Level 0Scout
0 XP0/41 lessons0/12 achievements
0/100 XP to next level100 XP to go0% complete
Quiz
01What is the defining property of a 'dense' model?
Hint
It's about *how much fires*, not how much exists.
02Llama 3.1 405B is one of the largest openly available dense models. Approximately how many parameters activate per token?
Hint
Read the lesson definition literally.
03Which efficiency innovation does Gemma 2/3 introduce *within* a dense backbone?
Hint
What changes is the attention shape, not whether the model is sparse.
04For which scenario is a dense model generally the BEST choice over MoE?
Hint
Where is the tooling most mature?
05Two dense models from different labs both have ~14B parameters and both use GQA + RoPE + SwiGLU. They feel different to use. The most likely axis the difference lives on is:
Hint
Same architecture, different recipe.
Comments 0
🔔 Reply notifications (sign in)Sign in — Please sign in to comment.
No comments yet — be the first.