AI-driven scientific progress threatened by an obsession
AI-driven scientific progress threatened by an obsession
A new US study finds that newer AI models reviewing scientific papers draw roughly 50–70% of their criticisms from appendices, compared with under 30% for GPT-5, and that appendix-related criticism in human peer review has risen to 27.6% after years below 11%. Testing 2,000 papers from ACL 2026, EMNLP 2026, COLM 2026 and ICLR 2027, the authors report that GPT-6.1-Sol and GPT-6-Astra show the strongest bias, which persists across API, Codex and web access methods, and that LLMs treat all text as equally important rather than recognizing the main body as the paper's core. The paper, titled 'AI Reviews Are Biased Towards the Appendix', warns that reviews focused on minor appendix details may not engage with a paper's main contribution, and its code is available on GitHub.
Source: Hacker News AI · unite.ai