SatBench releases a benchmark of AI benchmarks: 109 of 173 are saturated
Understand this story
simonkk has released SatBench, which treats the 173 benchmarks most frequently cited by AI labs as an exam, with a best reported score of 85% considered solved. As of data from November 14, 2026, 109 benchmarks have been solved, for overall progress of 63%. The index stayed around 30% for a long stretch from 2023 to early 2025, and has risen 20 percentage points in the last 12 months; the share of new benchmarks solved within a year of release rose from 17% in 2023 to 43% in 2025. SatBench also gives a stricter measure: counting only independent reproduction runs, progress is 24% rather than 63%. In addition, a benchmark must be mentioned 10 times in release posts to enter SatBench, which takes months, so benchmarks released in 2026, most of which are not yet solved, are not all counted yet; Artificial Analysis scores were not included due to terms restrictions. SatBench is built on Brian Zhou's Benchmark Almanac.
AI overview based on source reports · Updated 19 min ago
Coverage timeline
Follow the coverage to explore different perspectives.
- Hacker News AISatBench a Benchmark of AI Benchmarks
SatBench treats the 173 benchmarks AI labs cite most as questions on one exam, where a question counts as solved once the best reported AI score reaches 85%, and finds 109 of 173 (63%) are solved, up from about 30% between 2023 and early 2025. Its own first edition of 44 benchmarks crossed 85% in September 2026, and the share solved within a year of release rose from 17% for 2023 releases to 43% for 2025. A companion analysis finds the 2023 benchmark canon is extinct at the frontier, with 2 of 134 frontier-lab launch posts in 2026 naming MMLU, HumanEval, GSM8K, MATH, BBH, HellaSwag, DROP or MGSM, and notes that counting only independent runs puts SatBench at 24% rather than 63%.
Activity for this story
Not enough continuous observations to show a trend yet.