arXiv cs.CL· Josh Barua, Kyle Mahowald, William Rudman·· 3 天前AI 评分27
视觉语言模型如何像有偏的贝叶斯观察者一样整合冲突证据
Vision-Language Models Integrate Conflicting Evidence Like Biased Bayesian Observers
AI 导读
研究从贝叶斯视角提出，视觉语言模型（VLM）会将来自文本和图像的条件独立证据与先验信念结合，并受模态特定偏差影响。所构建的两参数线索组合模型能准确预测 VLM 在未见过的输入退化组合下的偏好，且无需重新拟合即可迁移到新的不可靠形式。该框架将证据强度与整合偏差分离，解释了 VLM 为何能在更重视视觉证据的同时仍跟随文本，从而在单一组合规则下统一了文本主导与视觉主导的报道。
正文 · 原文
Abstract:Vision-language models must reconcile conflicting text and images. Prior work has characterized the resulting behavior in terms of modality biases--e.g., a "blind faith in text"--but why preferences vary across models, domains, and input conditions remains unclear. We develop a predictive account from a Bayesian perspective, positing that VLMs combine conditionally independent evidence from each modality with prior beliefs, subject to modality-specific biases. The resulting two-parameter cue combination model accurately predicts VLM preferences under unseen combinations of input degradation and transfers to new forms of unreliability without refitting. Separating evidence strength from integration bias explains how VLMs can follow text even while weighting visual evidence more heavily, reconciling reports of text and vision dominance under a single combination rule.
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV) |
| Cite as: | arXiv:2610.19407 [cs.CL] |
| (or arXiv:2610.19407v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.19407 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Josh Barua [view email]
[v1]
Wed, 14 Oct 2026 01:55:36 UTC (2,157 KB)
来源：arXiv cs.CL · arxiv.org