Study: Over Half of AI Review Criticism Targets Paper Appendices
Understand this story
A new US study tested 2,000 papers from ACL 2026, EMNLP 2026, COLM 2026, and ICLR 2027, and found that newer AI review models concentrate most of their criticism on paper appendices rather than the main text. The study said that for newer models such as GPT-6.1-Sol and GPT-6-Astra, about 50%–70% of criticism came from appendices, compared with less than 30% for GPT-5. The study concluded that these models tend to focus more on secondary supporting details rather than the paper's main findings.
AI overview based on source reports · Updated 25 min ago
Coverage timeline
Follow the coverage to explore different perspectives.
- Hacker News AIAI-driven scientific progress threatened by an obsession
A new US study finds that newer AI models reviewing scientific papers draw roughly 50–70% of their criticisms from appendices, compared with under 30% for GPT-5, and that this appendix bias appears to be spreading into human peer review, where appendix-related criticism rose to 27.6% in the latest reviews. Testing 2,000 papers from ACL 2026, EMNLP 2026, COLM 2026 and ICLR 2027, the authors found the bias persisted across API, Codex and web-interface access, with AI reviewers more than seven times as likely as humans to flag appendix weaknesses. The paper, titled "AI Reviews Are Biased Towards the Appendix," warns that attention to minor details can come at the expense of a paper's main findings.
Activity for this story
Not enough continuous observations to show a trend yet.