Together AI Blog·· 2 days agoAI score58
Making open models work for long-horizon agentic workloads
Making open models work for long-horizon agentic workloads
AI summary
Ollama and Together AI explain how agentic workloads are dominated by long input sequences and short outputs, making prefix caching essential, and detail how harness choices (late tool loading, changing system prompts, reasoning effort changes, oversized tool results, subagent context) and engine choices (cache-aware routing, offloading idle caches, PagedAttention, continuous batching, prefill/decode disaggregation) must both protect the cache to keep long sessions fast.
Source: Together AI Blog · together.ai