Two fundamentally different memory models
Attention and SSMs aren't "two algorithms for the same operation". They're two genuinely different ways to handle sequence memory. The clearest way to see it: attention is a database, SSMs are a stream summary.
An attention layer at position i pulls explicit per-token weights against every prior token's value. The full n×n attention matrix is the database — every cell is a direct connection between two tokens, and the model can route information from any past token to the current one if it wants. An SSM at position i has access only to the current input x_i and the current state h_i. The state h_i is whatever survived being squeezed through the recurrence; there is no path back to a specific token at position j<i.
The strength side
SSMs win on resource consumption. Constant memory at inference. Linear compute. No KV-cache to grow. Tractable on consumer hardware at sequence lengths where Transformers OOM. For workloads where most of the relevant information is captured in the state — streaming inference over audio, time-series forecasting, long-form text where the bulk content is what matters — this is fantastic.
The weakness side
SSMs lose on tasks that need precise retrieval of specific past tokens. Show an SSM a long passage and then ask "what was the third number in the second paragraph?" — pure SSMs systematically fail. The information was there, but the recurrence's job is summarization, not indexing. The 2025 NeurIPS paper "Achilles' Heel of Mamba" made this rigorous: pure SSMs have provable failure modes on associative recall that Transformers don't share, and these failure modes can't be patched with more data — they're structural.
This is the core trade. If you ever find yourself comparing an SSM to a Transformer on a single benchmark and asking which one is "better", you're asking the wrong question. The right question is: does my workload care about exact recall over the full window, or about steady summarization at low cost?