LiteLLM and Revenium budget limit stress test exposes overspending and lost records
Understand this story
On October 29, 2026, Peter Olayemi and Onatunde Oladimeji stress-tested per-customer budget limits on LiteLLM v1.106.6 and Revenium Python SDK 0.12.0, covering concurrent bursts, oversized responses, backend failures, and 250 concurrent customers; they concluded that overspending and lost records occur under default configurations. Most tests set the per-customer budget at $0.05, which about 35 calls could exceed; OpenAI usage across all tests cost less than $2. The authors said they reported the SDK 0.12.0 configuration issue to Revenium on October 30, along with reproduction steps and logs. The authors packaged the tests into a free open-source tool called stipend-check.
AI overview based on source reports · Updated 1 hr ago
Coverage timeline
Follow the coverage to explore different perspectives.
- Hacker News AIWe stress-tested AI spend limits under bursts and outages
The authors tested LiteLLM v1.106.6 and Revenium Python SDK 0.12.0 in October 2026 with a $0.05 per-customer budget on gpt-4o-mini, finding that a well-configured LiteLLM held budgets to within about 2.5% even through a database outage, while several of its safest behaviors are off by default. Revenium's accounting matched the authors' tally to within $0.00004, but its enforcement works from a copy of rules refreshed on a timer, so overshoot grew with spending rate and reached 117% at the 60-second default refresh. The piece lists recommended settings for both tools and notes Revenium fixed an enforcement URL issue in SDK 0.12.1, released October 31, 2026.
Activity for this story
Not enough continuous observations to show a trend yet.