vLLM optimizes DeepSeek-V4.1-Flash inference on Vera Rubin NVL72
Understand this story
The Inferact and vLLM teams say they deployed DeepSeek-V4.1-Flash on a single NVIDIA Vera Rubin NVL72 rack, achieving per-GPU throughput of 1.7M tokens/s and 387 tokens/s per user, 58 times that of GB300 NVL72; peak throughput is 2.52M tokens/s, with a per-user maximum of 1,810 tokens/s. The team says this deployment uses the same technology stack as the earlier one on GB300 NVL72, in which the HBM pool is still experimental.
AI overview based on source reports · Updated 2 days ago
Coverage timeline
Follow the coverage to explore different perspectives.
- vLLM BlogFeaturedDeepSeek-V4.1-Flash on Vera Rubin NVL72: 58x Throughput vs. GB300 NVL72 at 387 tokens/s
On a single NVIDIA Vera Rubin NVL72 rack, vLLM serves DeepSeek-V4.1-Flash at 1.7M tokens/s per GPU with 387 tokens/s per user, 58x GB300 NVL72, reaching 2.52M at peak throughput and 1,810 tokens/s for a single user. The gains come from kernels tuned for Rubin's split memory domains and FP4 tensor cores, specialized low-latency decode kernels, and a rack-wide KV cache and Engram tables shared over NVLink. The post details four optimization groups and reports a 96.5–97% prefix hit rate at high concurrency with workload-aware routing.
Activity for this story
- Current comparable activity
- 3
- Peak comparable activity
- 5Nov 13 08:00
- Change in the past 24 hours
- –
Trends compare the same fully observed participants, which may cover fewer participants than current activity. Move the pointer or tap the chart to inspect hourly activity; use the left and right arrow keys with a keyboard.