Psychologist uses Claude to audit his own 79 papers
Understand this story
A Stanford developmental psychologist used Claude Code (Opus 5.5) to audit 79 papers he had authored over the past five years, 59 of which were empirical papers. The results showed that 58 had code and data publicly available, but only 31 could be fully rerun using the public data and code; 40 had at least one number that could not be reproduced within a 10% tolerance, and more than half of the papers were found to contain at least one error. An upstream bug in one model training pipeline may require corrections or retractions for the related papers.
AI overview based on source reports · Updated 3 days ago
Coverage timeline
Follow the coverage to explore different perspectives.
- Hacker News AIA developmental psychologist audits his own research with AI
A developmental psychologist used Claude Code to audit 79 papers he co-authored over five years, finding that 58 of 59 empirical papers had some code and data posted but only 31 could be fully rerun, and that at least one error of some type was identified in more than half of the papers, including one substantial upstream bug that may warrant correction or retraction. The audit also surfaced privacy lapses, with identifiers and persistent IDs reaching repositories, which he says have been remediated and reported to the appropriate data protection office. He calls the exercise disturbing and proposes routine AI-assisted pre-submission audits and automated repository checks.
Activity for this story
- Current comparable activity
- 3
- Peak comparable activity
- 10Nov 12 01:00
- Change in the past 24 hours
- -50%
Trends compare the same fully observed participants, which may cover fewer participants than current activity. Move the pointer or tap the chart to inspect hourly activity; use the left and right arrow keys with a keyboard.