C.W.K.
Stream
Lesson 04 of 05 · published

Nemotron-H and Enterprise Scale

~12 min · nemotron-h, nvidia, enterprise

Level 0Observer
0 XP0/50 lessons0/14 achievements
0/100 XP to next level100 XP to go0% complete

The largest hybrid SSM-attention model

NVIDIA Nemotron-H (April 2025) is the canonical enterprise-scale hybrid: 8B and 56B / 47B versions, with ~92% of attention layers replaced by Mamba-2. About 3× faster than Llama-3.1 70B at 65K context. The 56B model was pretrained in FP8 on 6,144 H100 GPUs using 20T tokens — production pretraining at frontier scale.

This is the data point that ends the "is hybrid SSM-attention production-ready" question. NVIDIA, the most hardware-aware company in the field, bet 6,144 H100s on a hybrid architecture and shipped it as their commercial offering. That's not a research bet; it's a serving bet.

MiniPuzzle compression — 56B → 47B

NVIDIA also showed that hybrid architectures are amenable to aggressive post-training compression. The 56B model was distilled / pruned to 47B parameters with a 20% speed gain on top of the architectural advantage. The compression worked partly because hybrid stacks have natural redundancy across SSM layers that pure Transformer stacks don't have to the same degree.

Cosmos-Reason 1 — vision-language application

Nemotron-H serves as the backbone for Cosmos-Reason 1, NVIDIA's physical AI vision-language model. The hybrid backbone is what makes Cosmos-Reason's long-context multi-modal reasoning practical — high-res video frames at scale require exactly the long-context efficiency that hybrids provide.

Nemotron 3 — the next generation

Nemotron 3 (December 2025 / March 2026) is NVIDIA's next-generation hybrid: MoE architecture, 1M context, and NVFP4 precision (a new low-precision format optimized for Blackwell GPUs). The combination of MoE sparsity, hybrid SSM-attention, and aggressive low-precision training represents NVIDIA's bet on what 2026–2027 production stacks look like.

The clear pattern: NVIDIA is committing more to hybrid SSM-attention with each generation, not less. That commitment from the hardware vendor itself is a signal worth taking seriously when planning longer-term architecture decisions.

External links

Exercise

Pull NVIDIA Nemotron-H 8B from Hugging Face. Generate output and time it against Llama 3.1 8B at 65K context. The reported 3× advantage at 65K should reproduce on commodity H100 hardware. If you don't have an H100, run on whatever you have and observe the relative scaling — even on a smaller GPU, the hybrid will pull ahead at long context.

Progress

Progress is local-only — sign in to sync across devices.
Spotted a bug or have feedback on this page?Report an Issue

Comments 0

🔔 Reply notifications (sign in)
Sign inPlease sign in to comment.

No comments yet — be the first.