arXiv cs.CL· Beth Pearson, Michael Wray, Martha Lewis·· 2 天前AI 评分22
视觉语言模型如何处理否定？研究用物体注意力对比人类注视
Probing Negation Understanding in Vision-Language Models With Object Attention
AI 导读
研究通过分析注意力模式，考察视觉语言模型（VLM）处理否定的机制，并对比人类注视与模型注意力在肯定句和否定句下的差异。结果显示模型处理否定时通常不遵循类人注意力模式，PaliGemma2-3B 在否定下会将物体注意力转向替代对象，但头级消融表明这只是副产物而非否定处理机制；LLaVA-1.5-7B 则表现出对左侧物体的空间偏好。下游任务评估显示所有模型在肯定与否定问题上均存在性能差距。
正文 · 原文
Abstract:Negation is a fundamental aspect of language understanding and it is important vision-language models (VLMs) are able to process it effectively. We investigate how VLMs process negation by analysing attention patterns of different models. We draw comparisons between human gaze and attention in models when processing affirmative vs negated statements. We compare object level model attention to human eye-fixations in both binary and multary settings, evaluating with both synthetic and realistic datasets. We find models generally do not follow human-like attention patterns when resolving negation, suggesting they rely on mechanisms outside of attention. While layerwise analysis shows that PaliGemma2-3B shifts object attention to the alternative under negation, our headwise ablations suggest this is a byproduct rather than a negation processing mechanism. Additionally, we find that LLaVA-1.5-7B displays a spatial bias towards left-side objects. Our evaluation of models on downstream tasks shows that all models have a discrepancy between affirmative and negated questions highlighting the need to better understand the underlying processing of negation to close this performance gap. Code available at this https URL
| Comments: | EMNLP Findings 2026 |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.20852 [cs.CV] |
| (or arXiv:2610.20852v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2610.20852 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Martha Lewis [view email]
[v1]
Wed, 14 Oct 2026 19:41:18 UTC (10,399 KB)
来源：arXiv cs.CL · arxiv.org