iCreatAI News
Skip to content
Trending storyWatching

SatBench releases AI benchmark saturation analysis

1 reports1 sources1 days agoUpdated

Understand this story

AI overview

simonkk has released the SatBench project, which treats the 173 benchmarks most frequently cited by AI labs as a single exam, with a best reported score of 85% considered "solved." By this measure, 63% (109) of the benchmarks are saturated; counting only independent reproductions, the saturation rate is 24%. SatBench says the index hovered around 30% from 2023 to early 2025 and rose 20 percentage points in the past 12 months; the share of benchmarks released in 2023 that were solved within a year was 17%, 30% for 2024, and 43% for 2025. The data is current as of November 14, 2026. SatBench also notes that data before 2025 is sparser and that its projections for the remaining benchmarks assume they are no harder than the benchmarks already solved.

AI overview based on source reports · Updated 1 days ago

Coverage timeline

Follow the coverage to explore different perspectives.

Nov 16
  1. Hacker News AI
    SatBench a Benchmark of AI Benchmarks

    SatBench treats the 173 benchmarks most cited by AI labs as questions on one exam, counting a question solved once the best reported AI score reaches 85%, and finds 109 of 173 (63%) solved, up from about 30% between 2023 and early 2025. A companion analysis finds the 2023 benchmark canon is extinct at the frontier, that nearly every benchmark correlates with every other (median rank correlation 0.67, falling to 0.06 after controlling for Epoch's Capabilities Index), and that counting only independent runs puts SatBench at 24% rather than 63%. The author cautions that many "solved" results rest on labs' own launch-post claims, pre-2025 data is thin, and the projections are extrapolations, not forecasts.

Activity for this story

Current comparable activity
4
Peak comparable activity
10Nov 17 00:00
Change in the past 24 hours
-49%

Trends compare the same fully observed participants, which may cover fewer participants than current activity. Move the pointer or tap the chart to inspect hourly activity; use the left and right arrow keys with a keyboard.