iCreatAI News
Skip to content
Hacker News AI· depthfirst·· 17 hr agoAI score63

Show HN: Company docs didn't help our AI agent. Four shortnotes doubled accuracy

Show HN: Company docs didn't help our AI agent. Four shortnotes doubled accuracy

AI summary

Running a Claude Haiku 5.5 agent on the DABstep payments dataset, the authors found company docs alone gave no accuracy gain over tables alone (36% vs 40% of tries correct on a held-out test), though they cut cost. Four short notes written from the agent's wrong answers raised accuracy to 76% at about a quarter of the cost per answer, and the gain held on 42 questions the notes were never written from.

Source: Hacker News AI · contextflo.com