iCreatAI News
Skip to content
Hacker News AI· amirmc·· 1 days agoAI score44

Challenges for human oversight in detecting AI-assisted diagnostic malfunctions

Challenges for human oversight in detecting AI-assisted diagnostic malfunctions

AI summary

In a randomized experiment with 48 physicians classifying heart sounds while AI reliability declined across four blocks (90%, 60%, 50%, 0%), combined human-AI performance fell below unassisted human performance under low reliability, and only 12.5% of participants reported noticing the declining reliability. Cognitive forcing (requiring an independent classification before seeing the AI recommendation) did not improve diagnostic performance; at the lowest reliability block the AI-First group outperformed the AI-Second group. The authors conclude that human oversight alone may be insufficient to reliably detect AI system malfunctions and call for system- and organizational-level safeguards.

Source: Hacker News AI · nature.com