Study: Doctors' Accuracy Drops When AI Stethoscope Malfunctions
Understand this story
A randomized controlled trial published in npj Digital Medicine found that doctors struggle to detect failures in AI cardiac auscultation systems. As the reliability of AI recommendations gradually dropped from 90% to 0%, the assisted diagnostic accuracy of 48 doctors fell from 81.45% to 58.33%. The study also tested whether cognitive forcing strategies (such as requiring doctors to make independent judgments before seeing AI recommendations) could improve performance, and concluded that they did not. At the lowest reliability stage, the AI-First group, which saw AI recommendations first, had an accuracy of 64.58%, actually higher than the 52.08% of the AI-Second group, which made independent judgments first. The paper's authors state that the study examined whether and when doctors can detect AI failures, intervene appropriately, and whether cognitive forcing strategies can help accomplish these tasks.
AI overview based on source reports · Updated 1 days ago
Coverage timeline
Follow the coverage to explore different perspectives.
- Hacker News AIChallenges for human oversight in detecting AI-assisted diagnostic malfunctions
In a randomized experiment with 48 physicians classifying heart sounds while AI reliability declined across four blocks (90%, 60%, 50%, 0%), combined human-AI performance fell below unassisted human performance under low reliability, and only 12.5% of participants reported noticing the declining reliability. Cognitive forcing (requiring an independent classification before seeing the AI recommendation) did not improve diagnostic performance; at the lowest reliability block the AI-First group outperformed the AI-Second group. The authors conclude that human oversight alone may be insufficient to reliably detect AI system malfunctions and call for system- and organizational-level safeguards.
Activity for this story
- Current comparable activity
- 4
- Peak comparable activity
- 10Nov 16 23:00
- Change in the past 24 hours
- -49%
Trends compare the same fully observed participants, which may cover fewer participants than current activity. Move the pointer or tap the chart to inspect hourly activity; use the left and right arrow keys with a keyboard.