The breadth of retention's domain reach
RetNet didn't ship as a frontier LLM, but it did spawn a remarkably broad set of domain-specific applications in 2024–2025. The retention primitive turned out to be useful in places nobody initially predicted:
- Vision: RMT, RetViT, SegRet — retention-based vision transformers competitive with attention-based ones.
- Audio: RetNet-EEND for end-to-end speaker diarization; some music-generation work.
- Scientific computing: JetRetNet for particle physics, EEG decoding work, haplotype assembly in genomics.
- Time-series: LeRet and other retention-based forecasting models.
The first comprehensive survey of retention applications across domains was published June 2025 (arXiv:2506.06708). The breadth signals that exponential-decay-based retention is a useful inductive bias for any temporally-structured signal — not just language.
Why no major LLM provider shipped one
Despite this breadth, no major frontier LLM provider has shipped a RetNet-based product. Anthropic, OpenAI, Google, Meta, Mistral — none have a retention-based flagship. Why?
The most honest answer: by the time RetNet's ideas matured (2024), the field was already moving toward Mamba-style selectivity. Mamba's input-dependent gating is a strict superset of retention's fixed decay (you can recover RetNet by making γ constant in Mamba), and Mamba had GPU-friendly kernels available earlier. RetNet got out-shipped by a closely related architecture that did the same job slightly better.
The domain pattern
Where RetNet did succeed is on workloads where the data has well-understood temporal structure and exact recall isn't critical. Vision (where spatial structure replaces temporal), audio (where the signal is genuinely fading), scientific data (where the noise model is well-understood). For general-purpose language modeling, retention got out-competed. For domain-specific work, it found a niche.