iCreatAI News
Skip to content
Trending storyUpdating

Together AI and Ollama Publish Long-Horizon Agent Cache Optimization Practices

1 reports1 sources23 hr agoUpdated

Understand this story

AI overview

Together AI and Ollama have jointly published a cache optimization practices guide for long-horizon coding agent workloads. The two parties state that such agents have long inputs and short outputs, making prefix caching a performance criticality; one of the focuses of their work is preventing one party's operations from breaking the cache that the other party relies on. The guide provides cache protection practices from both the harness (agent runtime framework) and the inference engine sides, aimed at agent and infrastructure developers.

AI overview based on source reports · Updated 36 min ago

Coverage timeline

Follow the coverage to explore different perspectives.

Nov 16
  1. Together AI Blog
    Making open models work for long-horizon agentic workloads

    Ollama and Together AI explain how agentic workloads are dominated by long input sequences and short outputs, making prefix caching essential, and detail harness-side pitfalls that break the cache—such as late-loaded tools, changing system prompts, reasoning-effort changes, oversized tool results, and subagent design—alongside engine-side needs like cache-aware routing, offloading idle caches, PagedAttention, continuous batching, and separating prefill and decode clusters.

Activity for this story

Not enough continuous observations to show a trend yet.