AI reads through astrophysics literature, recomputes 659 papers and finds 872 errors
Understand this story
Caltech theoretical astrophysics PhD Casey Handmer used AI to read through 158,059 papers and 554,670 citations in the field of theoretical astrophysics, and line-by-line recomputed 659 highly cited papers, confirming 872 errors distributed across 433 papers. Handmer says these errors have not yet been manually verified. Handmer says buying model time at list price would cost about $2,720, equivalent to about $3 per confirmed error; the actual run was completed within a Claude Max subscription. The entire project (including other fields) consumed about 4.9 billion tokens, and he personally invested about 44 hours over 41 days. The deep recomputation currently covers only 4% of the highly cited paper graph, proceeding from highest to lowest citation count.
AI overview based on source reports · Updated 2 days ago
Coverage timeline
Follow the coverage to explore different perspectives.
- Hacker News AII had AI read my entire field, Theoretical Astrophysics
An astrophysicist used AI to screen 158,059 papers and 554,670 citations in theoretical astrophysics, fully screening 50,269 papers and recomputing 659 of the most important, verifying 872 errors in 433 papers. The analysis found that 367 papers copied an error while only 68 criticised one, that peer review removed roughly one error in five, and that half of copied errors were still being copied seven years later. The author estimates about 34,000 undetected errors in papers cited 100 or more times, and notes the deep check has reached only 4% of highly cited papers on the map.
Activity for this story
- Current comparable activity
- 3
- Peak comparable activity
- 10Nov 18 14:00
- Change in the past 24 hours
- -51%
Trends compare the same fully observed participants, which may cover fewer participants than current activity. Move the pointer or tap the chart to inspect hourly activity; use the left and right arrow keys with a keyboard.