arXiv cs.CL· Amanuel Gizachew Abebe, Yasmin Moslem·· 12 小时前AI 评分24
SpanCalib-VLM：视觉语言模型中的校准幻觉片段检测
SpanCalib-VLM: Calibrated Hallucination Span Detection in Vision-Language Models
AI 导读
SpanCalib-VLM 提出一种双系统混合方案用于视觉语言模型的幻觉片段检测，将 XLM-RoBERTa-Large 与 SigLIP 视觉编码器经交叉注意力融合的序列标注器，与微调生成式 VLM（Qwen3.5-4B-SHROOM-SFT）结合，通过 Union-Calibrated Fusion 用标注器的校准概率对生成模型候选片段重新打分。
正文 · 原文
Abstract:Detecting hallucinations in Large Vision-Language Models (LVLMs) requires both accurate span localization and well-calibrated confidence scores. Fine-tuned generative VLMs excel at identifying hallucinated text spans but suffer from overconfidence and high inference latency. Discriminative sequence taggers offer deterministic speed and superior calibration but exhibit conservative span recall. We present SpanCalib-VLM, a hybrid dual-system for the SHROOM-Visions Shared Task that combines a multimodal sequence tagger, consisting of XLM-RoBERTa-Large fused with a SigLIP vision encoder via cross-attention, with our fine-tuned generative VLM (Qwen3.5-4B-SHROOM-SFT). Through a Union-Calibrated Fusion strategy, candidate spans from the generative model are re-scored with calibrated probabilities from the sequence tagger. On the SHROOM-Visions English validation split, our ensemble achieves a Pearson calibration correlation of 0.41 and an overall IoU of 0.39 (with a clean-response IoU of 0.91 and overall detection accuracy of 70.7%). On the official blind test set, SpanCalib-VLM achieves a Pearson correlation of 0.32 and an IoU of 0.25 on English, substantially outperforming the shared task baseline (0.15 on both metrics) and demonstrating consistent gains across all four languages. We make our model weights and code publicly available.
| Comments: | Shroom-Visions (UncertainLP) |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL) |
| Cite as: | arXiv:2608.29974 [cs.CV] |
| (or arXiv:2608.29974v2 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2608.29974 arXiv-issued DOI via DataCite |
Submission history
From: Yasmin Moslem [view email]
[v1]
Sun, 30 Aug 2026 19:00:31 UTC (31 KB)
[v2]
Mon, 19 Oct 2026 18:14:10 UTC (32 KB)
来源：arXiv cs.CL · arxiv.org