C.W.K.
Stream
Lesson 04 of 05 · published

Strengths, Limitations, and the Recall Wall

~12 min · limitations, recall, language-modeling

Level 0Observer
0 XP0/50 lessons0/14 achievements
0/100 XP to next level100 XP to go0% complete

Where Hyena shines

Hyena is at its best on workloads that look like the genomics workload: very long sequences, patterns that are more regular than natural language, and tolerance for some loss of exact retrieval. Specifically:

  • Genomic modeling — DNA, RNA, protein sequences. Evo 2 is the canonical example. HyenaDNA was the predecessor.
  • Byte-level modeling — operating directly on bytes (no tokenizer), where sequences are 4× longer than tokenized form.
  • Time-series forecasting on data with strong periodic structure, where convolutional inductive bias helps.
  • Long-context document processing where the task is statistical pattern capture, not retrieval.

Where Hyena struggles

On natural language modeling at standard scale, Hyena is competitive but doesn't dominate. At 360M scale, Hyena has perplexity 10.11 vs. attention's 8.39 — close but a real gap. At larger scales the gap shrinks but doesn't fully close, and Mamba-family architectures have generally pulled ahead on language tasks.

The biggest practical limitation is recall. On recall-intensive benchmarks, Hyena scores around 5.1 compared to attention's 47.7 — that is, Hyena is roughly an order of magnitude worse at retrieving specific past tokens. This is even weaker than Mamba on the same family of tasks, because Hyena's convolution-based mixing has no equivalent of selectivity. Filters are generated from positions, not from content.

Training fragility

Hyena is notably learning-rate sensitive — narrower stable LR window than even Mamba. The implicit-filter FFNs are sensitive to initialization, and convergence can be brittle if you're not following the published recipes carefully. For research labs this is a tax; for production teams it's a barrier. This is part of why Hyena hasn't seen broader language-modeling adoption beyond the StripedHyena line.

External links

Exercise

Pick a recall-intensive benchmark (the LongBench QA tasks or a synthetic key-value retrieval set) and run the StripedHyena-7B checkpoint against it. Compare scores against a Llama-2 7B baseline. Then run the same checkpoints on a long-context summarization task (e.g., GovReport). The relative scores will flip — Hyena loses on QA, often wins or ties on summarization. That's the recall-vs-summarization axis, in numbers.

Progress

Progress is local-only — sign in to sync across devices.
Spotted a bug or have feedback on this page?Report an Issue

Comments 0

🔔 Reply notifications (sign in)
Sign inPlease sign in to comment.

No comments yet — be the first.