Together AI and Ollama Publish Long-Horizon Agent Cache Optimization Practices
Understand this story
Together AI and Ollama have jointly published a cache optimization practices guide for long-horizon coding agent workloads. The two parties state that such agents have long inputs and short outputs, making prefix caching a performance criticality; one of the focuses of their work is preventing one party's operations from breaking the cache that the other party relies on. The guide provides cache protection practices from both the harness (agent runtime framework) and the inference engine sides, aimed at agent and infrastructure developers.
AI overview based on source reports · Updated 36 min ago
Coverage timeline
Follow the coverage to explore different perspectives.
- Together AI BlogMaking open models work for long-horizon agentic workloads
Ollama and Together AI explain how agentic workloads are dominated by long input sequences and short outputs, making prefix caching essential, and detail harness-side pitfalls that break the cache—such as late-loaded tools, changing system prompts, reasoning-effort changes, oversized tool results, and subagent design—alongside engine-side needs like cache-aware routing, offloading idle caches, PagedAttention, continuous batching, and separating prefill and decode clusters.
Activity for this story
Not enough continuous observations to show a trend yet.