arXiv cs.CL· Soroosh Tayebi Arasteh, Mehdi Joodaki, Mahshad Lotfinia, Tri-Thien Nguyen, Sebastian Ziegelmayer, Sven Nebelung, Daniel Truhn·· 11 小时前AI 评分42
BRACE：自动验证 AI 胸片陈述的 grounding 框架
Automated grounding verification of AI statements about chest radiographs
AI 导读
研究提出 BRACE 框架，可从公开放射学报告和 Radiopaedia 文章中自动构建训练样本，同时核查报告与引用证据。在 5 个公开胸片数据集上对比 0.1-284B 参数的 42 个验证器，BRACE 训练后的验证器在 0.90 支持样本阈值下，仅接受 0.239 被报告反驳的 AI 陈述，而未用矛盾样本训练的验证器平均接受 0.830。
正文 · 原文
Abstract:A grounding verifier checks a statement about a patient's image against the radiology report and the cited evidence. The state-of-the-art methods for testing these verifiers remove or replace the cited evidence while the patient's report stays unchanged. To our knowledge, no existing method checks both the report and the cited evidence. We present Building Report And Citation Examples (BRACE), an automated framework that builds training examples from public radiology reports and Radiopaedia articles. We compared a lexical baseline and 42 verifiers with 0.1-284B parameters on 5 different public chest radiograph datasets. At a threshold accepting 0.90 of the supported test examples, verifiers trained with BRACE accepted a mean of only 0.239 of AI-written statements contradicted by the report. Verifiers trained without contradicted examples passed the state-of-the-art tests, yet accepted most of these statements (0.830 on average). With citations retrieved from held-out Radiopaedia articles, verifiers trained with BRACE accepted 0.169 of AI-written statements with an agreeing report and an unsupporting citation. By contrast, zero-shot large language models mostly checked the report and not the citation (they accepted 0.830 of these statements). We further assessed the framework manually, with two board-certified radiologists independently labeling a sample of test reports. The framework's labels agreed with each radiologist (k > 0.910) almost as closely as the radiologists agreed with each other (k = 0.952). Moreover, instantiating the framework for food allergens in recipes, we showed that it extends beyond clinical applications. A verifier built the same way accepted 0.493 of the statements contradicted by the recipe, whereas a lexical baseline accepted 0.898. Using both the report and the cited evidence, verifiers trained with BRACE could check AI statements before clinicians read them.
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Information Retrieval (cs.IR); Machine Learning (cs.LG) |
| Cite as: | arXiv:2604.09537 [cs.CL] |
| (or arXiv:2604.09537v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2604.09537 arXiv-issued DOI via DataCite |
Submission history
From: Soroosh Tayebi Arasteh [view email]
[v1]
Fri, 10 Apr 2026 17:55:38 UTC (2,758 KB)
[v2]
Fri, 30 Oct 2026 19:02:17 UTC (1,293 KB)
来源：arXiv cs.CL · arxiv.org