vLLM launches FlyDSL MonoKernel to reduce AMD decoding latency
Understand this story
The official vLLM blog introduced a MonoKernel decoding path written with AMD FlyDSL: it merges multiple operators within the decoding layer into a small number of persistent kernel launches, reducing decoding latency for interactive agents on MI355X, MI350X, and MI325X. This path must be actively enabled by adding --monokernel to a supported recipe; if the deployment is unsupported, vLLM will refuse to start and list all the reasons. Ports for GLM-5.x, Qwen3.8, and MiniMax-M3.1 are still under review, and wider steps use the original path.
AI overview based on source reports · Updated 2 days ago
Coverage timeline
Follow the coverage to explore different perspectives.
- vLLM BlogKeeping Agents Fast: High-Interactivity Decode with FlyDSL MonoKernels in vLLM
This post describes how MonoKernels written in FlyDSL, AMD's Python DSL and MLIR compiler for ROCm kernels, reduce decode latency in vLLM for interactive LLM serving on AMD Instinct GPUs by combining dependent operator stages into persistent single-kernel launches with thread-block reuse, weight prefetch, and tagged-mailbox handoffs. Mono decode is opt-in via --monokernel and merged in vLLM main for DeepSeek-V4.1-Flash, DeepSeek-V4-Pro, MiniMax-M3, Kimi-K3, and Muse Spark on MI355X (plus MI325X for DeepSeek-V4.1-Flash), reporting concurrency-1 speedups of 1.20× to 2.26× with accuracy differences within noise for the listed configurations. The post also notes limits: gains are largest at low concurrency, step widths are capped (e.g., 48 rows for DeepSeek-V4.1-Flash on MI355X), and MI300X/MI325X keep separate weight copies.
Activity for this story
- Current comparable activity
- 3
- Peak comparable activity
- 7Nov 18 23:00
- Change in the past 24 hours
- -49%
Trends compare the same fully observed participants, which may cover fewer participants than current activity. Move the pointer or tap the chart to inspect hourly activity; use the left and right arrow keys with a keyboard.