arXiv cs.AI· Bruce Changlong Xu, Jose K. James, Alexander J. Ryu·· 13 小时前AI 评分31
从理论到决策规则：在三个医学影像基准上校准视觉语言模型弱监督的噪声标签交叉点
From Theory to Decision Rule: Calibrating the Noisy-Label Crossover for Vision-Language Model Weak Supervision Across Three Medical-Imaging Benchmarks
AI 导读
研究在 PatchCamelyon、ISIC 2019 和 NIH ChestX-ray14 三个医学影像基准上，用 BiomedCLIP 与 PubMedCLIP 生成的伪标签配合 8M–89M 参数的六种学生架构。
正文 · 原文
Abstract:In medical imaging, small sets of expert labels are often combined with zero-shot pseudo-labels from a vision-language model (VLM). When the expert-labeled set is very small, the extra pseudo-labels reduce variance and improve accuracy. Once a classifier trained on expert labels alone reaches the accuracy of the weak labeler, however, adding pseudo-labels lowers test performance. We measure where this crossover occurs on PatchCamelyon histopathology, ISIC 2019 dermoscopy, and NIH ChestX-ray14 radiography using pseudo-labels from BiomedCLIP and PubMedCLIP across six student architectures ($8\mathrm{M}$ to $89\mathrm{M}$ parameters). Weak labels stop helping after $20$--$50$ expert labels on ISIC 2019, about $100$ on PatchCamelyon, and $250$--$500$ on NIH ChestX-ray14. Above those budgets, adding $10{,}000$ pseudo-labels reduces test macro-AUC by up to $0.14$ points with BiomedCLIP ($0.23$ with PubMedCLIP) on ISIC 2019, $0.10$ on PatchCamelyon, and $0.08$ on NIH ChestX-ray14. Across the six student architectures, the weak-label ceiling stays within a $0.024$ AUC range ($0.841$--$0.865$), indicating that the VLM labeler, rather than student size, limits performance. Comparing VLM pseudo-labels with random label flips at the same error rate shows that systematic VLM mistakes yield $0.09$--$0.20$ lower AUC than random noise on PatchCamelyon and ISIC 2019, but $0.03$--$0.04$ higher AUC on 15-class NIH ChestX-ray14, where per-finding crossovers also span more than a $20\times$ range. A simple bias-variance rule estimated from the weak labeler's validation accuracy and small expert-only runs predicts whether pseudo-labels will help or hurt in all $30$ statistically significant settings ($39/42$ overall).
| Comments: | 25 pages, 3 figures, 12 tables. Accepted at the ICML 2026 Workshop on Combining Theory and Benchmarks (CTB) |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2605.24771 [cs.CV] |
| (or arXiv:2605.24771v2 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2605.24771 arXiv-issued DOI via DataCite |
Submission history
From: Bruce Xu [view email]
[v1]
Sat, 23 May 2026 23:15:07 UTC (103 KB)
[v2]
Tue, 20 Oct 2026 22:48:22 UTC (134 KB)
来源：arXiv cs.AI · arxiv.org