arXiv cs.CL· Reza Khanmohammadi, Kundan Thind, Mohammad M. Ghassemi·· 13 小时前AI 评分31
适应还是校准？局部标签如何在医疗 VQA 中换取可认证覆盖率
Adapt or Calibrate? How Local Labels Buy Certified Coverage in Medical VQA
AI 导读
研究在 Learn-then-Test 框架下考察局部标签能为九个置信度估计器在五个开源 VLM 和三个医疗 VQA 基准上认证什么。自然图像训练的估计器迁移效果差，最佳合并证书在 20% 误差容忍下仅覆盖 11.5% 的 SLAKE 答案，PATH-VQA 上为零。
正文 · 原文
Abstract:An answer from a vision-language model (VLM) to a medical image question can be accepted without review only under a threshold bounding the error of accepted answers, and certifying that bound costs local labels. We ask what such labels can certify for nine confidence estimators on five general-purpose open-weight VLMs and three medical VQA benchmarks under Learn-then-Test, with the question as the exchangeable unit. Estimators trained on natural images transfer poorly: the best pooled certificate covers 11.5% of SLAKE answers at a 20% error tolerance and nothing on PATH-VQA. With 1,000 labeled SLAKE questions, adapting a probe on half of them raises pooled certified coverage from 6.8% to 37.6% (0.0% to 30.6% with the image as the unit), above the best threshold-only estimator (11.3%). On PATH-VQA, adaptation certifies 11.4% with 2,500 labels; thresholds alone certify nothing. Yet for strong single models, small budgets are better spent on the threshold. A question-only classifier recovers much of adaptation's ranking gain but certifies far less. Neither procedure yields a pooled certificate at 10%, and nothing is certified at 5%. A blank image removes 61% of correct answers, yet on GMAI-MMBench no trained probe is less confident on answers unchanged by image replacement.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2606.15910 [cs.CL] |
| (or arXiv:2606.15910v3 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2606.15910 arXiv-issued DOI via DataCite |
Submission history
From: Reza Khanmohammadi [view email]
[v1]
Sun, 14 Jun 2026 16:39:17 UTC (138 KB)
[v2]
Thu, 20 Aug 2026 05:21:27 UTC (137 KB)
[v3]
Fri, 23 Oct 2026 18:14:26 UTC (93 KB)
来源：arXiv cs.CL · arxiv.org