arXiv cs.CL· Guowei Zhong, Junjie Li, Huaiyu Zhu, Ruohong Huan, Yun Pan·· 13 小时前AI 评分18
不确定性引导的多模态共识（UMC）用于多模态情感分析
Uncertainty-Guided Multimodal Consensus for Multimodal Sentiment Analysis
AI 导读
研究者提出不确定性引导的多模态共识方法 UMC，先恢复连续的单模态伪标签，再在模态间形成可靠性感知的共识。其核心包含连续原型引导标签恢复（CPLR）与可靠性与精度共识（RPC）两部分，后者用闭式共识求解器结合预测统计与跨模态分歧估计样本级模态可靠性。在四个公开 MSA 数据集上的实验显示该方法有效且优于对比模型。
正文 · 原文
Abstract:Multimodal sentiment analysis (MSA) has made considerable progress in recent years. Existing MSA methods mainly learn shared multimodal representations or train unimodal predictors with a common multimodal label. However, assigning the shared label to every modality can obscure modality-specific sentiment, while direct fusion may overemphasize unreliable predictions when modalities disagree. To address these issues, we propose Uncertainty-Guided Multimodal Consensus (UMC), which first recovers continuous unimodal pseudo-labels and then forms a reliability-aware consensus across modalities. Our method consists of two key components: i) Continuous Prototype-Guided Label Recovery (CPLR), which combines heteroscedastic predictions with ordered sentiment prototypes and uses confidence and label conflict to refine continuous unimodal pseudo-labels, and ii) Reliability and Precision Consensus (RPC), which estimates sample-dependent modality reliability from predictive statistics and cross-modal disagreement and combines it with predictive precision through a closed-form consensus solver. Extensive experiments conducted on four publicly available MSA datasets demonstrate the effectiveness and superiority of our proposed model.
| Comments: | Submitted to WWW2027 |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL); Machine Learning (cs.LG); Multimedia (cs.MM) |
| Cite as: | arXiv:2510.20256 [cs.CV] |
| (or arXiv:2510.20256v2 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2510.20256 arXiv-issued DOI via DataCite |
Submission history
From: Guowei Zhong [view email]
[v1]
Thu, 23 Oct 2025 06:25:10 UTC (1,208 KB)
[v2]
Tue, 27 Oct 2026 06:14:16 UTC (370 KB)
来源：arXiv cs.CL · arxiv.org