iCreatAI News
Skip to content
Trending storyWatching

vLLM launches FlyDSL MonoKernel to reduce AMD decoding latency

1 reports1 sources3 days agoUpdated

Understand this story

AI overview

The official vLLM blog introduced a MonoKernel decoding path written with AMD FlyDSL: it merges multiple operators within the decoding layer into a small number of persistent kernel launches, reducing decoding latency for interactive agents on MI355X, MI350X, and MI325X. This path must be actively enabled by adding --monokernel to a supported recipe; if the deployment is unsupported, vLLM will refuse to start and list all the reasons. Ports for GLM-5.x, Qwen3.8, and MiniMax-M3.1 are still under review, and wider steps use the original path.

AI overview based on source reports · Updated 2 days ago

Coverage timeline

Follow the coverage to explore different perspectives.

Nov 18
  1. vLLM Blog
    Keeping Agents Fast: High-Interactivity Decode with FlyDSL MonoKernels in vLLM

    This post describes how MonoKernels written in FlyDSL, AMD's Python DSL and MLIR compiler for ROCm kernels, reduce decode latency in vLLM for interactive LLM serving on AMD Instinct GPUs by combining dependent operator stages into persistent single-kernel launches with thread-block reuse, weight prefetch, and tagged-mailbox handoffs. Mono decode is opt-in via --monokernel and merged in vLLM main for DeepSeek-V4.1-Flash, DeepSeek-V4-Pro, MiniMax-M3, Kimi-K3, and Muse Spark on MI355X (plus MI325X for DeepSeek-V4.1-Flash), reporting concurrency-1 speedups of 1.20× to 2.26× with accuracy differences within noise for the listed configurations. The post also notes limits: gains are largest at low concurrency, step widths are capped (e.g., 48 rows for DeepSeek-V4.1-Flash on MI355X), and MI300X/MI325X keep separate weight copies.

Activity for this story

Current comparable activity
3
Peak comparable activity
7Nov 18 23:00
Change in the past 24 hours
-49%

Trends compare the same fully observed participants, which may cover fewer participants than current activity. Move the pointer or tap the chart to inspect hourly activity; use the left and right arrow keys with a keyboard.