arXiv cs.CL· Kai Qian, Dongjie Xu, Weijie Shi, Yezhao Xu, Jiaqi Wang, Hanghui Guo, Mengze Li, Hao Chen, Ziyi Liu, Jiajie Xu·· 13 小时前AI 评分22
HMDR：面向缺失模态假新闻检测的模态头解耦与调控方法
Decoupling and Regulating Modality Heads for Fake News Detection under Missing Modality
AI 导读
针对多模态新闻在传播中丢失图像或文本、原有真实性标签不再可靠的问题，研究者提出 Head-wise Modality Decoupling and Regulation（HMDR），通过按注意力偏好软分组解耦模态表示、补偿不平衡模态注意力来调控各头贡献，并用模态不变性学习对齐单模态表示。两个基准实验显示，HMDR 在缺失模态下提升检测性能，同时在完整输入上保持原有表现。
正文 · 原文
Abstract:Multimodal news often loses its image or text as it spreads across Web platforms. Existing methods treat this as information loss, assuming that the original veracity label remains valid. This assumption fails for fake news reusing authentic material, whose remainder may be genuine. Detectors should therefore decouple modality cues and adapt their contributions to modality availability. However, coarse news labels provide little guidance for modality decoupling, while training on complete inputs can induce modality bias. To address these challenges, we propose Head-wise Modality Decoupling and Regulation (HMDR) for multimodal large language models. First, Modality Head Decoupling decouples modality representations through soft head grouping by attention preference. Second, Modality Head Regulation regulates head contributions through compensation for imbalanced modality attention. Finally, Modality Invariance Learning promotes invariance by aligning representations of each modality under replacement of the other. Experiments on two benchmarks show that HMDR improves detection under missing modalities while maintaining performance on complete inputs.
| Comments: | 10 pages. Revised manuscript; updated title and author list |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL) |
| Cite as: | arXiv:2604.09711 [cs.CV] |
| (or arXiv:2604.09711v2 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2604.09711 arXiv-issued DOI via DataCite |
Submission history
From: Kai Qian [view email]
[v1]
Wed, 8 Apr 2026 04:13:55 UTC (1,606 KB)
[v2]
Tue, 27 Oct 2026 03:00:21 UTC (327 KB)
来源：arXiv cs.CL · arxiv.org