iCreatAI News
Skip to content
Hacker News AI· simonkk·· 1 days agoAI score58

SatBench a Benchmark of AI Benchmarks

SatBench a Benchmark of AI Benchmarks

AI summary

SatBench treats the 173 benchmarks most cited by AI labs as questions on one exam, counting a question solved once the best reported AI score reaches 85%, and finds 109 of 173 (63%) solved, up from about 30% between 2023 and early 2025. A companion analysis finds the 2023 benchmark canon is extinct at the frontier, that nearly every benchmark correlates with every other (median rank correlation 0.67, falling to 0.06 after controlling for Epoch's Capabilities Index), and that counting only independent runs puts SatBench at 24% rather than 63%. The author cautions that many "solved" results rest on labs' own launch-post claims, pre-2025 data is thin, and the projections are extrapolations, not forecasts.

Source: Hacker News AI · github.com