arXiv cs.CL· Ziyin Zhang, Yuyu Guo, Ying Zhang, Siqiao Xue, Zihan Liao, Jin Qin, Wenjie Yang, Siyuan Yang, Shijie Li, Cheng Chen, Hao Wu, Zhaoyang Zhu, Ruipeng Li, Zhaokun Jiang, Lei Lei, Rui Wang, Hang Yu, Rui Wang·· 11 小时前AI 评分31
从碎片化到融合：全模态嵌入研究综述与路线图
From Fragmentation to Fusion: A Survey and Roadmap for Omnimodal Embeddings
AI 导读
一篇立场论文提出全模态嵌入研究正因对比学习、Transformer 架构与粒度转变而走向收敛，并给出覆盖、可比、组合三级能力框架。对应三条统一路径为变换扩大覆盖、对齐实现可比、联合嵌入指向组合性，作者将联合嵌入视为终点。论文称这是首个对全模态嵌入研究的统一跨领域梳理。
正文 · 原文
Authors:Ziyin Zhang, Yuyu Guo, Ying Zhang, Siqiao Xue, Zihan Liao, Jin Qin, Wenjie Yang, Siyuan Yang, Shijie Li, Cheng Chen, Hao Wu, Zhaoyang Zhu, Ruipeng Li, Zhaokun Jiang, Lei Lei, Rui Wang, Hang Yu, Rui Wang
Abstract:Embedding models underpin virtually every modern AI system, from search engines to recommendation systems. Despite shared mathematical foundations, research on embedding models remains fragmented by modality: the communities develop largely in isolation, seldom exchanging insights. We argue that this fragmentation obscures a deeper convergence. Driven by contrastive learning, Transformer architectures, and a granularity shift, embedding research across modalities is converging toward a common set of principles, even as modality-specific challenges remain underexplored. To reason across this convergence without conflating it, we propose a framework of three progressively harder capabilities: coverage, comparability, and compositionality. Three paths toward unification satisfy these in turn: transformation broadens coverage, alignment enables comparability, and joint embedding, the terminus we advocate, aims at compositionality. This position paper traces the resulting evolution, providing the first unified, cross-cutting treatment of omnimodal embedding research.
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.00172 [cs.CL] |
| (or arXiv:2611.00172v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2611.00172 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Ziyin Zhang [view email]
[v1]
Fri, 30 Oct 2026 14:07:34 UTC (3,726 KB)
来源：arXiv cs.CL · arxiv.org