SatBench releases AI benchmark saturation analysis
Understand this story
simonkk has released the SatBench project, which treats the 173 benchmarks most frequently cited by AI labs as a single exam, with a best reported score of 85% considered "solved." By this measure, 63% (109) of the benchmarks are saturated; counting only independent reproductions, the saturation rate is 24%. SatBench says the index hovered around 30% from 2023 to early 2025 and rose 20 percentage points in the past 12 months; the share of benchmarks released in 2023 that were solved within a year was 17%, 30% for 2024, and 43% for 2025. The data is current as of November 14, 2026. SatBench also notes that data before 2025 is sparser and that its projections for the remaining benchmarks assume they are no harder than the benchmarks already solved.
AI overview based on source reports · Updated 1 days ago
Coverage timeline
Follow the coverage to explore different perspectives.
- Hacker News AISatBench a Benchmark of AI Benchmarks
SatBench treats the 173 benchmarks most cited by AI labs as questions on one exam, counting a question solved once the best reported AI score reaches 85%, and finds 109 of 173 (63%) solved, up from about 30% between 2023 and early 2025. A companion analysis finds the 2023 benchmark canon is extinct at the frontier, that nearly every benchmark correlates with every other (median rank correlation 0.67, falling to 0.06 after controlling for Epoch's Capabilities Index), and that counting only independent runs puts SatBench at 24% rather than 63%. The author cautions that many "solved" results rest on labs' own launch-post claims, pre-2025 data is thin, and the projections are extrapolations, not forecasts.
Activity for this story
- Current comparable activity
- 4
- Peak comparable activity
- 10Nov 17 00:00
- Change in the past 24 hours
- -49%
Trends compare the same fully observed participants, which may cover fewer participants than current activity. Move the pointer or tap the chart to inspect hourly activity; use the left and right arrow keys with a keyboard.