arXiv cs.CL· Joey Guo, Rayan Ansari, Alaa Youssef·· 11 小时前AI 评分40
CheXPoT：对 VLM 生成的放射科报告进行事后视觉验证与纠正
CheXPoT: Post-Hoc Visual Verification and Correction of VLM-Generated Radiology Reports
AI 导读
CheXPoT 是一个程序化思维（PoT）框架，可逐条核验 VLM 生成的放射科报告发现是否有图像依据，并删除无法被图像支撑的内容。在 4,145 张图像的留出集上，MedGemma-4B 接入后微平均精确率从 19.2% 升至 46.8%，微平均 F1 从 23.0% 升至 49.4%；LLaVA-Med 和 MAIRA-2 上同样有提升。
正文 · 原文
Abstract:Vision-language models (VLMs) have shown promise for chest radiograph report generation, but adoption remains limited because of hallucinated findings that are not grounded in the image. Correcting these hallucinations and improving overall report quality remains a significant challenge. Prior work has addressed this problem by refining models with reinforcement learning, preference optimization, or chain-of-thought supervision, often at substantial compute cost. To address this gap, we introduce CheXPoT, a program-of-thought (PoT) framework that verifies each generated finding against image features and removes those it cannot ground. CheXPoT decomposes generated reports into predicates with RadGraph and an LLM, grounds these predicates against a DenseNet-121 and Grad-CAM visual evidence table, executes PoT verification code, and rewrites reports after removing unsupported predicates. On a 4,145-image held-out split, CheXPoT yields statistically significant precision gains on 8 of 14 CheXpert classes and F1 gains on 5, with no significant decreases in either metric. CheXPoT lifts micro-average precision from 19.2% (MedGemma-4B) to 46.8% (MedGemma-4B + CheXPoT) and micro-average F1 from 23.0% to 49.4%, absolute gains of 27.6% and 26.4% respectively. We also tested CheXPoT on other VLM report generators. On LLaVA-Med, CheXPoT lifts micro-average precision from 4.5% to 17.7% and micro-average F1 from 6.3% to 19.4%. On MAIRA-2, CheXPoT lifts micro-average precision from 15.6% to 42.1% and micro-average F1 from 21.0% to 45.3%. Our results suggest that inexpensive post-hoc visual verification may improve VLM trustworthiness by correcting unsupported findings from generated reports. CheXPoT lays a foundation for tools that help validate VLM-generated radiology reports and reduce unsupported findings. These innovations may subsequently help increase trust and adoption of VLM tools in Medicine.
| Comments: | 15 pages, 12 figures |
| Subjects: | Computation and Language (cs.CL); Computer Vision and Pattern Recognition (cs.CV) |
| Cite as: | arXiv:2611.00785 [cs.CL] |
| (or arXiv:2611.00785v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2611.00785 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Joey Guo [view email]
[v1]
Sat, 31 Oct 2026 09:57:35 UTC (1,352 KB)
来源：arXiv cs.CL · arxiv.org