iCreatAI News
Skip to content
Trending storyUpdating

Baseten adds deferred tool loading for open models

1 reports1 sources18 hr agoUpdated

Understand this story

AI overview

Baseten has added deferred tool loading on its Model APIs for open models such as DeepSeek, Kimi, and GLM, supporting the Chat Completions and Messages API formats. Baseten says this mechanism reduces coding agent costs by 62.5%. The feature is also available via API to organizations building their own internal harnesses, but Baseten's stated condition for applicability is that users need to build context-focused internal harnesses.

AI overview based on source reports · Updated 17 hr ago

Coverage timeline

Follow the coverage to explore different perspectives.

Nov 20
  1. Baseten BlogFeatured
    Deferred tool loading: Make your agent 62.5% cheaper without changing your model or harness

    Baseten added deferred tool loading to open models (DeepSeek, Kimi, GLM) on its Model APIs, supporting Chat Completions and Messages API formats. In Claude Code with three MCPs enabled, coding benchmarks were on average 62.5% cheaper and 26.2% faster, while agentic benchmarks were 51.1% cheaper and 14.9% faster, with tool schemas dropping from about 141k tokens to about 1.3k. The technique keeps tools out of the system prompt and gives the model a tool_search tool that renders schemas inline at the bottom of the transcript, preserving the KV cache.

Activity for this story

Current comparable activity
6
Peak comparable activity
10Nov 20 03:00
Change in the past 24 hours
–

Trends compare the same fully observed participants, which may cover fewer participants than current activity. Move the pointer or tap the chart to inspect hourly activity; use the left and right arrow keys with a keyboard.