Baseten adds deferred tool loading for open models
Understand this story
Baseten has added deferred tool loading on its Model APIs for open models such as DeepSeek, Kimi, and GLM, supporting the Chat Completions and Messages API formats. Baseten says this mechanism reduces coding agent costs by 62.5%. The feature is also available via API to organizations building their own internal harnesses, but Baseten's stated condition for applicability is that users need to build context-focused internal harnesses.
AI overview based on source reports · Updated 17 hr ago
Coverage timeline
Follow the coverage to explore different perspectives.
- Baseten BlogFeaturedDeferred tool loading: Make your agent 62.5% cheaper without changing your model or harness
Baseten added deferred tool loading to open models (DeepSeek, Kimi, GLM) on its Model APIs, supporting Chat Completions and Messages API formats. In Claude Code with three MCPs enabled, coding benchmarks were on average 62.5% cheaper and 26.2% faster, while agentic benchmarks were 51.1% cheaper and 14.9% faster, with tool schemas dropping from about 141k tokens to about 1.3k. The technique keeps tools out of the system prompt and gives the model a tool_search tool that renders schemas inline at the bottom of the transcript, preserving the KV cache.
Activity for this story
- Current comparable activity
- 6
- Peak comparable activity
- 10Nov 20 03:00
- Change in the past 24 hours
- –
Trends compare the same fully observed participants, which may cover fewer participants than current activity. Move the pointer or tap the chart to inspect hourly activity; use the left and right arrow keys with a keyboard.