NVIDIA preprint: agents rarely need long context
Understand this story
NVIDIA researchers released a preprint, "You don't need long context," testing AI agent performance after context compression. All authors are NVIDIA employees. The study progressively lowered the context limit from 128K to 16K and compressed it to 12K, and Claude Opus 4.5 and GPT 5.6 Sol showed limited accuracy changes across 13 long-horizon and long-input benchmarks. Cost conclusions vary by model: at current nominal API prices, compression reduced Opus 4.5's cost on SWE-bench Verified from $420.00 to $295.75, but raised GPT 5.6 Sol's from $167.68 to $188.96. The study also estimates model pricing using a cloud service cost of $3.85 per H100 hour and an "optimized retail cost" that scales API prices with context length.
AI overview based on source reports · Updated 2 days ago
Coverage timeline
Follow the coverage to explore different perspectives.
- Hacker News AINvidia Study: AI Agents Rarely Need Long Context Windows
An NVIDIA preprint by Belcak, Molchanov and co-authors reports that Claude Opus 4.5 and GPT 5.6 Sol lost little to no accuracy when their context was compacted into short windows across 13 long-horizon and long-input benchmarks, with accuracy shifts of −3.7 to +5.6 points for Opus 4.5 and −3.3 to +4.4 points for GPT 5.6 Sol on long-horizon tasks. Serving a 256K window cost 3.6× to 7× more per output token than a 16K window on open-weight models measured with vLLM on H100 GPUs, while all tested open models lost SWE-bench Verified accuracy at the tightest setting, with Qwen 3 Coder Next collapsing from 53.0% to 3.6%. The authors note the headline savings rely on their own price model rather than real bills, results come from single runs without confidence intervals, and all authors work at NVIDIA.
Activity for this story
- Current comparable activity
- 3
- Peak comparable activity
- 10Nov 4 23:00
- Change in the past 24 hours
- -49%
Trends compare the same fully observed participants, which may cover fewer participants than current activity. Move the pointer or tap the chart to inspect hourly activity; use the left and right arrow keys with a keyboard.