arXiv cs.CL· Longzhen Han, Awes Mubarak, Almas Baimagambetov, Nikolaos Polatidis, Thar Baker·· 12 小时前AI 评分26
多模态生成模型综述：六大生成类别与四类技术
A Survey of Generative Categories and Techniques in Multimodal Generative Models
AI 导读
一篇综述将多模态生成模型按图像、音乐、视频、人体动作、3D 物体等六类文本条件生成模态分类，并考察自监督学习、MoE、RLHF 与 CoT 提示四类技术在各类模态上的适配方式。综述围绕忠实性、组合性与鲁棒性组织评估，指出偏好提升未必改善文本相关性、奖励优化可能降低同一提示内的多样性，专家粒度会改变机制及存储、计算与运行时成本。文章还分析了偏见、隐私泄露与生成媒体滥用等可信风险及缓解策略证据。
正文 · 原文
Abstract:Multimodal generative models (MGMs) have rapidly expanded beyond text generation to encompass images, music, video, human motion, and 3D objects. This survey categorises six primary text-conditioned generative modalities and examines how four techniques, namely self-supervised learning, mixture of experts, reinforcement learning from human feedback, and chain-of-thought prompting, are adapted across them, distinguishing shared mechanisms from experimentally supported benefits. We organise evaluation around faithfulness, compositionality, and robustness, and interpret evidence within each study's protocol. Selected within-study comparisons reported in the reviewed literature support four findings: preference gains need not improve text relevance, and reward optimisation can reduce within-prompt diversity; intermediate plans require tests of their realisation in final outputs; expert granularity changes the mechanism and its storage, computation, and runtime costs; and gains against an optimisation score need confirmation outside that scoring path. We further analyse trustworthiness risks, including bias, privacy leakage, and misuse of generated media, and examine the evidence for mitigation strategies.
| Comments: | Revised version; corresponding author named |
| Subjects: | Multimedia (cs.MM); Artificial Intelligence (cs.AI); Computation and Language (cs.CL) |
| Cite as: | arXiv:2506.10016 [cs.MM] |
| (or arXiv:2506.10016v4 [cs.MM] for this version) | |
| https://doi.org/10.48550/arXiv.2506.10016 arXiv-issued DOI via DataCite |
Submission history
From: Longzhen Han [view email]
[v1]
Thu, 29 May 2025 12:29:39 UTC (2,356 KB)
[v2]
Fri, 13 Jun 2025 20:56:42 UTC (2,356 KB)
[v3]
Mon, 24 Nov 2025 16:26:13 UTC (2,516 KB)
[v4]
Tue, 20 Oct 2026 08:59:15 UTC (1,224 KB)
来源：arXiv cs.CL · arxiv.org