iCreatAI News
Skip to content
Together AI Blog·· 23 hr agoAI score58

Making open models work for long-horizon agentic workloads

Making open models work for long-horizon agentic workloads

AI summary

Ollama and Together AI explain how agentic workloads are dominated by long input sequences and short outputs, making prefix caching essential, and detail harness-side pitfalls that break the cache—such as late-loaded tools, changing system prompts, reasoning-effort changes, oversized tool results, and subagent design—alongside engine-side needs like cache-aware routing, offloading idle caches, PagedAttention, continuous batching, and separating prefill and decode clusters.

Source: Together AI Blog · together.ai