The clear cases where SSM/hybrid wins
SSM / hybrid architectures are worth tracking — and possibly adopting — when you have one of these conditions:
- You need 128K–1M+ token contexts on limited hardware. SSMs process 220K+ tokens on a 24GB GPU vs. ~25K for pure Transformers. If your workload requires long context AND your serving budget is tight, this is the regime where alternatives are clearly compelling.
- Inference throughput is your binding constraint. 3–8× speedup at long context is real and consistent across published deployments. If you're memory-bandwidth-bound or throughput-bound at long context (and not quality-bound), SSM/hybrid earns its complexity.
- Memory footprint matters for deployment. Constant per-token memory at inference enables on-device, embedded, and edge deployments that growing-KV-cache Transformers can't reach without aggressive caching strategies.
- Streaming or real-time inference benefits from constant-state architectures because there's no warmup cost as context grows.
Start with hybrids, not pure SSMs
If you're going to bet on the SSM family, the safest move is a proven hybrid architecture: Jamba 1.5 Mini, NVIDIA Nemotron-H 8B, IBM Granite 4.0, or Bamba-9B. These give you most of the SSM efficiency benefits while keeping enough attention layers to maintain Transformer-like quality on recall-heavy tasks.
Pure SSMs (Falcon Mamba 7B, Mamba-2.8B base) are interesting research artifacts but are higher-risk for production: lower recall ceiling, narrower training stability if you fine-tune, and a less mature serving story. Reach for them only when memory or compute constraints make even sparse-attention-bearing hybrids impractical.
The shortlist to evaluate in 2026
- Jamba 1.5 Mini (12B) — friendly entry point, 256K context, available on major clouds + Hugging Face open weights.
- NVIDIA Nemotron-H 8B — open weights, vLLM support, ~3× faster than Llama 3.1 70B at 65K context.
- IBM Bamba-9B — day-0 vLLM integration, competitive quality-efficiency tradeoff.
- IBM Granite 4.0 (multiple sizes) — production-grade, 9:1 ratio, ~70% RAM reduction.
- Cartesia Llamba-8B — distilled from Llama 3.1 8B, 12× throughput; useful proof of the distillation route.