Study: Over Half of Criticism from New AI Review Models Targets Appendices
Understand this story
A new U.S. study tested 2,000 papers from ACL 2026, EMNLP 2026, COLM 2026, and ICLR 2027, finding that about 50%–70% of the criticism from new models such as GPT-6.1-Sol and GPT-6-Astra targeted paper appendices, compared with less than 30% for GPT-5. The study states that these models tend to focus more on secondary supporting details in papers rather than the main findings.
AI overview based on source reports · Updated 1 days ago
Coverage timeline
Follow the coverage to explore different perspectives.
- Hacker News AIAI-driven scientific progress threatened by an obsession
A new US study finds that newer AI models reviewing scientific papers draw roughly 50–70% of their criticisms from appendices, compared with under 30% for GPT-5, and that appendix-related criticism in human peer review has risen to 27.6% after years below 11%. Testing 2,000 papers from ACL 2026, EMNLP 2026, COLM 2026 and ICLR 2027, the authors report that GPT-6.1-Sol and GPT-6-Astra show the strongest bias, which persists across API, Codex and web access methods, and that LLMs treat all text as equally important rather than recognizing the main body as the paper's core. The paper, titled 'AI Reviews Are Biased Towards the Appendix', warns that reviews focused on minor appendix details may not engage with a paper's main contribution, and its code is available on GitHub.
Activity for this story
- Current comparable activity
- 4
- Peak comparable activity
- 10Nov 16 23:00
- Change in the past 24 hours
- -51%
Trends compare the same fully observed participants, which may cover fewer participants than current activity. Move the pointer or tap the chart to inspect hourly activity; use the left and right arrow keys with a keyboard.