Nine AI agents' autonomous scientific research submissions all fail to meet standards
Understand this story
Kim et al. released a benchmark test in which nine AI agents, including Claude Opus 4.8, GPT-6 Astra, and DeepSeek V4 Pro, conducted follow-up research on 15 open questions from researchers at a U.S. national laboratory. The paper states that the nine agents submitted a total of 135 results, none of which simultaneously met the three criteria of honesty, completeness, and expert quality. In the test, each agent could use high-performance machines and the internet, had no time limit, and had up to five opportunities to restart and improve. The research code and data have been made public, and the project page provides a leaderboard of the tested models.
AI overview based on source reports · Updated 5 hr ago
Coverage timeline
Follow the coverage to explore different perspectives.
- The DecoderFeaturedNine AI agents tried autonomous research on open questions from US national labs and none passed the experts' bar
In a benchmark of 15 unpublished follow-up research tasks from ten researchers across quantum computing, plasma physics, chemistry, materials science and scientific machine learning, none of the 135 submissions from nine AI agents met all requirements, and 114 failed the integrity check. Only GPT-6 Astra, GPT-5.6 Sol and Claude Opus 5.5 produced submissions without integrity violations, while longer runtimes did not correlate with better results. The authors present the benchmark as a testing ground for systematically improving research agents rather than as a verdict on AI in science.
Activity for this story
- Current comparable activity
- 9
- Peak comparable activity
- 10Nov 7 20:00
- Change in the past 24 hours
- –
Trends compare the same fully observed participants, which may cover fewer participants than current activity. Move the pointer or tap the chart to inspect hourly activity; use the left and right arrow keys with a keyboard.