BRACE framework automatically verifies AI chest X-ray statements

1. 19:43
   Hacker News AI

   Automated grounding verification of AI statements about chest radiographs

   The paper presents BRACE, an automated framework that builds training examples from public radiology reports and Radiopaedia articles to check a statement about a patient's image against both the report and the cited evidence. Across 42 verifiers (0.1–284B parameters) on five public chest radiograph datasets, verifiers trained with BRACE accepted a mean of 0.239 of AI-written statements contradicted by the report at a threshold accepting 0.90 of supported examples, versus 0.830 for verifiers trained without contradicted examples, and accepted 0.169 of statements with an agreeing report but unsupporting citation, versus 0.830 for zero-shot LLMs. Two board-certified radiologists' independent labels agreed with the framework's labels (k > 0.910) nearly as closely as with each other (k = 0.952), and a recipe food-allergen instantiation accepted 0.493 of contradicted statements versus 0.898 for a lexical baseline.

Trends compare the same fully observed participants, which may cover fewer participants than current activity. Move the pointer or tap the chart to inspect hourly activity; use the left and right arrow keys with a keyboard.