Kimi Linear — the headline 2025 result
Kimi Linear (Moonshot AI, October 2025) is the first linear-attention variant to claim across-the-board superiority over full Multi-head Latent Attention (MLA) at production scale. Headline numbers: 75% KV-cache reduction, 6× faster decoding at 1M context, and competitive or better quality on standard benchmarks. The architecture uses KDA (Key-Dependent Attention), a variant where the linear-attention state is updated in a way that depends on the keys (a kind of selectivity inside linear attention).
If the Kimi Linear results hold up at independent third-party evaluation, this is the moment linear attention finally caught up to full attention on quality while preserving its efficiency advantage. The Performer's old failure mode (quality degradation at long context) appears to be solved by smarter state design, not by better random features.
MHLA — the global-context-collapse fix
One of the recurring failure modes of linear-attention models is global context collapse: as you process more tokens, the running state averages over too much, and information about specific tokens fades into mush. MHLA (Multi-Head Linear Attention, January 2026) addresses this by partitioning tokens into blocks and maintaining local KV summaries that preserve broader context structure.
MHLA is a concrete pattern that's likely to recur: if the issue with linear attention is that all history blends together, structure the history into blocks so the blending is explicit and preserves locality. Variants of this pattern show up in Mamba's MIMO formulation and in the chunkwise modes of RetNet — different solutions to the same structural concern.
Gated Attention — the small win that compounds
Gated Attention (NeurIPS 2025 Best Paper) showed that even minimal modifications to standard attention — adding a simple gating mechanism on the attention output — yield measurable accuracy gains at minimal cost. The lesson: there's still meaningful headroom in refining attention itself, not just in finding alternatives to it.
This is the unsung half of the post-Transformer story. The other half is the dramatic alternatives. Both are real, and a healthy team should be aware of both lanes.