C.W.K.
Stream
Lesson 04 of 05 · published

Real-World Deployment — The Windows 10/11 Story

~12 min · deployment, windows, on-device

Level 0Observer
0 XP0/50 lessons0/14 achievements
0/100 XP to next level100 XP to go0% complete

The largest post-Transformer deployment in the world

Most discussion of post-Transformer architectures is research-flavored. RWKV has a single fact that breaks the frame: RWKV v5 (Eagle) was deployed to approximately 1.5 billion Windows 10 / 11 machines for on-device Copilot. By orders of magnitude this is the largest production deployment of any non-Transformer language-model architecture in the world, and most people in the alternative-architecture space don't realize it.

Why RWKV won that slot: on-device serving is the workload where constant-memory inference matters most. The Windows machines vary wildly in RAM — some have 4 GB, some have 64 GB. A Transformer with a growing KV-cache makes the memory budget unpredictable. RWKV's fixed-state RNN form gives Microsoft a flat memory profile they can plan against. The architectural decision was driven by deployability.

Performance numbers worth knowing

RWKV-G1 7.2B achieves around 145+ tokens/s at batch 1 on consumer hardware — and that throughput is constant regardless of context length, because there is no KV-cache to grow. For interactive on-device assistants, that flat curve is the whole product.

Community and ecosystem

RWKV is a Linux Foundation AI & Data project (member since February 2024), Vision-RWKV was accepted at ICLR 2025 Spotlight, and there are RWKV ports in roughly 30 languages — C, Rust, Go, Java, the lot. The model runs on Android and iOS via ONNX/GGML pipelines. The ecosystem is small compared to Transformer's but is unusually broad for an alternative architecture, and it skews heavily toward edge / on-device use cases.

Practical implication: if you're building anything that has to run client-side without network round trips — a desktop app's local assistant, a phone-side summarizer, a gaming companion — RWKV is the alternative architecture that is most likely to already be packaged for your target.

External links

Exercise

Find an RWKV ONNX or GGUF build for a 1–3B parameter checkpoint and run it on your phone or laptop using a community runtime (rwkv.cpp, ONNX Runtime, etc.). Generate text and confirm the per-token latency is flat as the prompt grows. Then try the same with a similar-size Llama checkpoint via llama.cpp and watch the prefill time grow. The RWKV experience is genuinely different — flat is flat.

Progress

Progress is local-only — sign in to sync across devices.
Spotted a bug or have feedback on this page?Report an Issue

Comments 0

🔔 Reply notifications (sign in)
Sign inPlease sign in to comment.

No comments yet — be the first.