Comfy Blog· rattus128·· 3 days agoAI score58
Doubling LLM token rates in ComfyUI
Doubling LLM token rates in ComfyUI
AI summary
ComfyUI rewrote the hot path of local text generation for Qwen3.x LLMs, letting the same checkpoints on the same GPU generate tokens up to twice as fast with no change to model or output quality; on an RTX 5090, Qwen3.5 9B went from 106 to 219 tokens per second. The changes include a re-laid-out GEMV kernel, fused prologue operations, new attention and DeltaNet decode kernels, and an adaptive-tree MTP speculative decoding algorithm, all available in ComfyUI v0.45.0 with the Text Generate node's mtp option left on auto.
Source: Comfy Blog · blog.comfy.org