arXiv cs.CL· Zhixin Cai, Ruoqi Yuan, Xinchen Yang, Zhuofan Chen, Jun Bai, Chuantao Yin, Wenge Rong·· 3 天前AI 评分20
ARP-CD：用注意力引导表征扰动缓解视频大模型幻觉
Mitigating Video Hallucinations via Attention-Guided Representation Perturbation
AI 导读
研究者提出免训练框架 ARP-CD，通过在表征空间构造对比负样本缓解 Video LLM 的视频幻觉：利用答案预测位置上视觉 token 的注意力质量，引导语言模型输入视觉隐状态做有界梯度更新，降低视觉注意力并生成负样本，与原始分支预测对比。在四个 Video LLM 和四个基准上，ARP-CD 在多数模型—基准组合中优于代表性免训练基线，减少幻觉的同时提升通用视频理解。
正文 · 原文
Abstract:Video large language models (Video LLMs) often generate responses unsupported by the input video. Contrastive decoding (CD) can mitigate these hallucinations, but its effectiveness depends on how the negative branch weakens visual evidence. Existing methods rely on predefined perturbations or masking guided by question reconstruction, offering limited control over the model's visual reliance during answer generation. We introduce Attention-guided Representation Perturbation for Contrastive Decoding (ARP-CD), a training-free framework for constructing contrastive negatives directly in representation space. ARP-CD uses the attention mass assigned to visual tokens at the answer-prediction position to guide bounded gradient updates of the language model's input visual hidden states. These updates reduce visual attention and produce a negative input, whose predictions are contrasted with those of the original branch throughout generation. Experiments across four Video LLMs and four benchmarks show that ARP-CD outperforms representative training-free baselines in most model--benchmark settings, reducing hallucinations while improving general video understanding. Further analyses show that attention-guided perturbation directions improve negative quality and that the resulting negatives provide effective contrastive references for correcting the original predictions.
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.19728 [cs.CV] |
| (or arXiv:2610.19728v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2610.19728 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Zhixin Cai [view email]
[v1]
Wed, 14 Oct 2026 07:50:08 UTC (3,631 KB)
来源：arXiv cs.CL · arxiv.org