vLLM supports Kimi K3.1 Pro on day one, covering four types of chips
Understand this story
vLLM announced that it completed inference support on the same day Moonshot AI's Kimi K3.1 Pro was released, covering NVIDIA GPUs, AMD GPUs, Google TPUs, and Alibaba T-Head Zhenwu PPUs. vLLM said this is the first time Zhenwu PPU has provided Day-0 support for a leading open-source large model, and that support will continue. K3.1 Pro is a 2.8T-parameter (104B activated) multimodal MoE open-weight model that adds DSA sparse attention to all 24 full-attention layers, with a 1M context window. vLLM noted that the currently supported version is a preview checkpoint, and the team is training a more accurate and efficient speculator that will be released separately once complete.
AI overview based on source reports · Updated 2 days ago
Coverage timeline
Follow the coverage to explore different perspectives.
- vLLM BlogFeaturedvLLM Delivers Day-0 Support for Kimi K3.1 Pro
vLLM announces day-0 support for Kimi K3.1 Pro, Moonshot AI's 2.8T-parameter (104B active) multimodal MoE with a 1M-token context that adds a DSA sparse-attention indexer to all 24 full-attention layers of its KDA + MLA hybrid. The post details a packed KV cache for three kinds of state, token-sharded prefill attention (up to 1.69× faster TTFT at 340K tokens on 8× GB300), decode context parallelism with an exact top-k merge, a fused low-latency decode path, a DSpark speculator preview, AgentX benchmark results on GB300 and Rubin, zero-mismatch RL, and support on NVIDIA, AMD, TPU and Alibaba T-HEAD Zhenwu PPUs.
Activity for this story
- Current comparable activity
- 3
- Peak comparable activity
- 6Nov 5 02:00
- Change in the past 24 hours
- -49%
Trends compare the same fully observed participants, which may cover fewer participants than current activity. Move the pointer or tap the chart to inspect hourly activity; use the left and right arrow keys with a keyboard.