arXiv cs.AI· Liangwei Nathan Zheng, Wenhao Liang, Wei Emma Zhang, Lin Yue, Weitong Chen·· 14 小时前AI 评分20
综述：用混合专家（MoE）应对多模态学习挑战
Tackling Multimodal Learning Challenges with Mixture-of-Expert: A Survey
AI 导读
一篇被 IJCAI 2026 接收的综述系统梳理了 MoE 如何解决多模态学习挑战，从三个视角展开：作为高效多模态引擎，将计算成本与参数增长解耦并通过选择性专家激活缓解模态冗余；作为多模态表示学习器，整合多意见专家知识以丰富对齐与交互表示；作为多模态适配器，以模块化机制应对模态不平衡与模态缺失。文章同时指出可解释路由、专家通信、模态融合与终身多模态学习等关键研究空白。
正文 · 原文
Abstract:Mixture-of-Experts (MoE) presents a naturally compatible and scalable framework for multimodal learning, demonstrating strong adaptability across diverse modalities and tasks. Despite its growing success, a comprehensive and systematic review on the MoE metho addressing multimodal challenges remains lacking. Existing surveys tend to evaluate either multimodal learning or MoE independently from method taxonomy, overlooking the unique interplay between them. This survey fills that gap by answering a central question: \textit{How does MoE effectively resolve multimodal challenges?} We approach this from three key perspectives: (1) \textbf{MoE as an Efficient Multimodal Engine:} enabling scalable multimodal modeling by decoupling computational cost from parameter growth and mitigating modality redundancy through selective expert activation; (2) \textbf{MoE as a Multimodal Representation Learner:} integrating complementary multi-opinion expert knowledge to enrich alignment and interaction representations; and (3) \textbf{MoE as a Multimodal Adapter:} providing a modular and flexible mechanism to model imperfect data scenarios such as modality imbalance and missing modality. Through our extensive literature review, we identify critical research gaps, including interpretable routing, expert communication, modality integration, and lifelong multimodal learning. We position this survey as a foundation for future research toward interpretable and sustainable multimodal Mixture-of-Experts system.
| Comments: | This survey has been accepted by IJCAI 2026. Results were released by 30 April 2026. This latest replacement is an extension version of survey to the previous version, which has been submitted to ACM Computing Survey for peer-review |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2605.27431 [cs.LG] |
| (or arXiv:2605.27431v2 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2605.27431 arXiv-issued DOI via DataCite |
Submission history
From: Liangwei Zheng [view email]
[v1]
Fri, 22 May 2026 05:01:21 UTC (232 KB)
[v2]
Wed, 28 Oct 2026 05:31:01 UTC (547 KB)
来源：arXiv cs.AI · arxiv.org