arXiv cs.CL· Hsin-Ling Hsu, Neeraja Kirtane, Kuan-Hao Huang·· 12 小时前AI 评分31
并非每个提及都是断言：重新审视视觉语言模型中的物体幻觉检测
Not Every Mention Is a Claim: Rethinking Object Hallucination Detection in Vision-Language Models
AI 导读
研究指出，现有视觉语言模型幻觉检测器把字幕中所有物体提及都当作“物体存在”的断言，而“a bench for people to sit on”这类非断言性提及会被误判为幻觉。
正文 · 原文
Abstract:Object hallucination in vision-language models (VLMs) is usually measured by checking each object a caption names against the image. Most benchmarks and detectors alike treat every object mention as a claim that the object is present. This assumption fails when a caption names an object without asserting it, as in "a bench for people to sit on", which mentions people without claiming that any are in the image. We call such mentions non-assertive. Treating them as hallucinations inflates the error rates attributed to models that use such language. To separate them from real hallucinations, we introduce NAME-Bench (Non-Assertive Mention Evaluation Benchmark), 25,497 object mentions from captions that five VLMs generate for COCO images. Each mention is labelled concrete (asserted and present), hallucinated (asserted and absent), or non-assertive, and non-assertive mentions are further divided into four subtypes: depictive, purpose, negation and hedged. All the hallucination detectors that we evaluate flag non-assertive mentions as hallucinations, at rates between 14% and 80%. Adding non-assertive mentions to detector training as negatives largely corrects this on LLaVA-1.5. On the four newer VLMs, a comparable correction takes five to ten times more data, and the detectors then catch fewer real hallucinations. These findings show that current detectors conflate mentioning an object with asserting it, and that this conflation is harder to undo on the four newer VLMs than on LLaVA-1.5. Hallucination evaluation should therefore separate what a caption asserts from what it merely mentions.
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.23610 [cs.CV] |
| (or arXiv:2610.23610v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2610.23610 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Neeraja Kirtane [view email]
[v1]
Fri, 16 Oct 2026 15:23:45 UTC (292 KB)
来源：arXiv cs.CL · arxiv.org