iCreatAI News
Skip to content
vLLM Blog· Inferact, Thinking Machines and NVIDIA·· 22 hr agoAI score58

Inkling 2 Mini on vLLM: Day-0 Support for a Looped Model

Inkling 2 Mini on vLLM: Day-0 Support for a Looped Model

AI summary

Thinking Machines announced Inkling 2 Mini, a multimodal MoE with 26B total and 3.5B active parameters, up to 1M tokens of context and a native DSpark speculator, whose middle 12 layers run three passes with shared weights, each pass keeping its own KV cache and reading gated messages from the pass before. vLLM supports it and the variable-loop research checkpoint TML-d24-varloop-research on day 0 on NVIDIA Blackwell, Hopper and DGX Spark, registering each extra pass as its own attention layer and fusing message reads, controllers and sublayer kernels. A Docker image, vllm/vllm-openai:inkling-2-mini, is provided, and vLLM serves the model for reinforcement learning with SkyRL.

Source: vLLM Blog · vllm.ai