Beyond the Transformer Quest

Tracks

1. 01🧭The Four Axes of Modern LLMs

   0/5 lessons

   Backbone, training, inference, product — the compass for everything else

   Every modern LLM lives at the intersection of four axes. Once you can name the axis a change lives on, you stop being fooled by marketing language that collapses them into one.

   Lesson list (5)

   • 01
     The Transformer Is Not the Whole Story~12 min · foundations, axes, literacy
   • 02
     The Four Axes of Modern LLMs~14 min · axes, framework, foundations
   • 03
     Total Params vs Active Params — The Number That Actually Matters~12 min · moe, parameters, axes
   • 04
     Three Lenses — Dense, MoE, Reasoning~10 min · overview, lenses, framework
   • 05
     Why Similar Parameter Counts Feel Different~10 min · intuition, parameters, performance

   ✓Quiz · 5 questions→

2. 02🧱Dense Models — The Default Frontier

   0/5 lessons

   Every token activates every parameter — the simplest thing that works

   Dense is the well-understood baseline. Llama, Gemma, Qwen, Mistral, Phi. Where dense still wins in 2026, what gives, and the families you should know.

   Lesson list (5)

   • 01
     What 'Dense' Actually Means~10 min · dense, fundamentals, ffn
   • 02
     The LLaMA Family — The Open-Weight Backbone of the Era~12 min · dense, llama, families
   • 03
     Gemma — Sliding Window Attention as a Dense Optimization~11 min · dense, gemma, attention
   • 04
     Qwen, Mistral, Phi — The Other Dense Families~10 min · dense, qwen, mistral, phi, families
   • 05
     When Dense Is Still the Right Choice~9 min · dense, decision, tradeoffs

   ✓Quiz · 5 questions→

3. 03🔀Mixture of Experts (MoE)

   0/6 lessons

   Many experts per layer, only a few fire per token — frontier capacity at affordable compute

   Not every token needs every expert. MoE breaks the equivalence between memory and compute, opening the door to trillion-parameter models you can actually serve.

   Lesson list (6)

   • 01
     The MoE Intuition — Not Every Token Needs Every Expert~12 min · moe, intuition, ffn
   • 02
     Routers, Top-K, and Shared Experts~11 min · moe, router, details
   • 03
     The Memory Paradox — Total in VRAM, Active in FLOPs~12 min · moe, memory, serving
   • 04
     MoE Families — Mixtral, DeepSeek, Llama 4, Qwen3~12 min · moe, families, deepseek, mixtral
   • 05
     Expert Load Balancing and Expert Collapse~11 min · moe, training, balancing
   • 06
     MoE Misconceptions and How to Read Past Them~9 min · moe, literacy, misconceptions

   ✓Quiz · 6 questions→

4. 04🧠Reasoning-Oriented Models

   0/5 lessons

   Same backbone, different training and inference — thinking longer for harder problems

   Reasoning is a behavior, not an architecture. RL post-training plus extended-thinking inference produces models that decompose, verify, and reach correct answers on problems that defeat one-shot generation.

   Lesson list (5)

   • 01
     Reasoning Is a Behavior, Not a Backbone~12 min · reasoning, axes, literacy
   • 02
     Test-Time Compute Scaling — What 'Thinking Longer' Actually Buys~12 min · reasoning, tts, inference
   • 03
     Visible, Hidden, and Summarized Thinking~9 min · reasoning, ux, transparency
   • 04
     How RL Creates Reasoning Behavior — SFT, RLHF, GRPO~14 min · reasoning, post-training, rl, grpo
   • 05
     Reasoning Model Families — o-series, Claude, Gemini, R1, Qwen~12 min · reasoning, families

   ✓Quiz · 6 questions→

5. 05🚫What Is NOT an Architecture

   0/6 lessons

   RAG, tools, agents, long context, speculative decoding — all wear architecture's coat, none of them are

   The most common confusion in LLM discourse is mistaking application layers for backbone changes. This track decodes the marketing phrasebook into the four axes underneath.

   Lesson list (6)

   • 01
     The Layered Stack — Backbone, Training, Inference, Product~10 min · framework, axes, literacy
   • 02
     RAG Is Not an Architecture~10 min · rag, product, framework
   • 03
     Tool Use and Function Calling Are Not Architecture~9 min · tools, product, framework
   • 04
     Agents Are Not an Architecture (and 'Long Context' Isn't Either)~10 min · agents, context, product
   • 05
     Speculative Decoding and KV-Cache Tricks Are Not Architecture~9 min · inference, speculative-decoding, kv-cache
   • 06
     The Marketing Decoder — A Phrasebook~9 min · marketing, literacy, framework

   ✓Quiz · 6 questions→

6. 06⚖️Real-World Comparison

   0/4 lessons

   Choosing dense, MoE, or reasoning per workload — and the cost-and-latency napkin math

   The decision table for production. Map your workload's bottleneck to the paradigm that addresses it. Estimate cost and latency before deploying.

   Lesson list (4)

   • 01
     Three Paradigms, One Decision Table~12 min · compare, decision, tradeoffs
   • 02
     Cost and Latency Napkin Math~11 min · compare, cost, latency
   • 03
     Decision Framework by Use Case~10 min · compare, decision, use-cases
   • 04
     How Architecture Choices Ripple Into Product Design~9 min · compare, product, ux

   ✓Quiz · 5 questions→

7. 07🔍Reading Model Announcements

   0/5 lessons

   Five questions, one phrasebook, no hype

   Apply the framework to every new release. Spot dense vs MoE from notation. Read benchmarks skeptically. Know what licenses actually allow. Read what is missing as carefully as what is present.

   Lesson list (5)

   • 01
     The Five Questions Every Builder Should Ask~10 min · literacy, announcements, reading
   • 02
     Spotting Dense vs MoE From Public Material~9 min · moe, literacy, announcements
   • 03
     Reading Benchmarks Skeptically~11 min · benchmarks, literacy, evaluation
   • 04
     Open Weight vs Open Source — License Literacy~9 min · licenses, literacy, open-source
   • 05
     Model Cards vs System Cards — What to Look For~9 min · model-cards, literacy, documentation

   ✓Quiz · 5 questions→

8. 08🔮The Emerging Frontier

   0/5 lessons

   Hybrids, Mamba, RWKV — what's actually next, and what's still the same

   Where the post-Transformer ideas are actually shipping (hybrids), where they aren't (pure SSMs at frontier scale), and how to read the frontier without drowning in hype.

   Lesson list (5)

   • 01
     Hybrid Futures — The Combinations Already Here~9 min · frontier, hybrid, future
   • 02
     Mamba and State-Space Models — Linear Scaling~12 min · frontier, mamba, ssm
   • 03
     RWKV — RNN-Like Inference at Transformer Quality~9 min · frontier, rwkv, rnn
   • 04
     Hybrid Architectures — The Real Convergence~11 min · frontier, hybrid, jamba
   • 05
     Why Transformers Still Dominate (and How to Stay Current)~10 min · frontier, transformers, literacy

   ✓Quiz · 5 questions→