arXiv cs.AI· Qiji Zhou, Junyi Hu, Peiyu Wang, Yue Zhang·· 6 天前AI 评分22
多模态 LLM 的 Evidence Anchoring Boundary：视觉干预在哪里有效、何时有用
The Evidence Anchoring Boundary: Where Visual Interventions Matter and When They Help in Multimodal LLMs
AI 导读
研究提出多模态大语言模型中的 Evidence Anchoring Boundary（EAB），即视觉残差编辑在解码器逐层施加时干预效果持续衰减的操作起点。在三个模型家族六个 checkpoint 上，EAB 之前留出评估查询的干预效果比之后大 13.78×–26.10×。
正文 · 原文
Abstract:Visual interventions provide a training-free way to reduce hallucinations in multimodal large language models. However, an intervention may have little leverage at some layers and may damage a correct answer even when it changes the output. We investigate where visual interventions retain leverage and when they improve predictions. First, we apply the same visual-residual edit at successive decoder layers, matching its RMS strength across depth. Across six checkpoints from three model families, we observe a persistent drop in intervention effect and define its operational onset for this edit as the \textbf{Evidence Anchoring Boundary (EAB)}. With boundaries fixed on calibration queries, held-out effects on disjoint evaluation queries before EAB are $13.78\times$--$26.10\times$ larger than those after it. Comparison with intermediate readouts shows that the edit can lose leverage before answer preferences stabilize. Second, we compare counterfactual subtraction across presence, attribute, and relation questions. Subtraction can correct a misleading preference, leave a decision unchanged, or remove a preference supporting the correct answer. Based on these observations, we develop calibrated selective routing, which preserves the original prediction unless calibration supports an intervention for the question type. It improves full-set attribute accuracy by $1.56$--$1.81$ percentage points across three checkpoints, while universal subtraction is neutral or harmful.
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.12981 [cs.CV] |
| (or arXiv:2610.12981v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2610.12981 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Qiji Zhou [view email]
[v1]
Fri, 9 Oct 2026 04:26:50 UTC (2,688 KB)
来源：arXiv cs.AI · arxiv.org