iCreatAI News
Skip to content
Comfy Blog· rattus128·· 3 days agoAI score58

Doubling LLM token rates in ComfyUI

Doubling LLM token rates in ComfyUI

AI summary

ComfyUI rewrote the hot path of local text generation for Qwen3.x LLMs, letting the same checkpoints on the same GPU generate tokens up to twice as fast with no change to model or output quality; on an RTX 5090, Qwen3.5 9B went from 106 to 219 tokens per second. The changes include a re-laid-out GEMV kernel, fused prologue operations, new attention and DeltaNet decode kernels, and an adaptive-tree MTP speculative decoding algorithm, all available in ComfyUI v0.45.0 with the Text Generate node's mtp option left on auto.

Source: Comfy Blog · blog.comfy.org