arXiv cs.AI· Libiao Chen, Xiyang Liu, Yanheng Wei, Zhenyu Tang·· 1 天前AI 评分28
UMER：用成对判别式推理统一嵌入与排序，实现通用多模态检索
UMER: Unifying Embedding and Ranking via Pair-Aware Discriminative Reasoning for Universal Multimodal Retrieval
AI 导读
UMER 提出统一多模态嵌入与排序框架，用成对判别式推理替代逐项推理，让单个 MLLM 同时学习用于全局匹配的对比嵌入和用于成对相关性判断的判别式排序。在 MMEB-V2 上，UMER-R 搭配 Qwen2-VL-2B/7B 分别得分 65.5/70.6，超过最强同骨干基线 1.4/2.0 分。该框架还支持从高效嵌入检索到可选成对重排序的预算可调推理。
正文 · 原文
Abstract:Modern Web information access increasingly involves heterogeneous multimodal content, including images, videos and visually rich documents. Users may seek not only globally similar content but also candidates that satisfy fine-grained attributes, relations or answer evidence under complex instructions. Supporting such access requires both efficient corpus-scale retrieval and fine-grained relevance reasoning. Multimodal large language model (MLLM)-based embedding methods derive representations from MLLM hidden states, and recent work further uses Chain-of-Thought (CoT) reasoning to improve embedding capability by injecting intermediate semantic evidence into the representation. However, existing CoT methods typically use item-wise reasoning over queries and candidates in isolation, providing no explicit evidence to distinguish a positive from a semantically confusable hard negative. Moreover, contrastive embeddings capture global similarity but struggle with meta-tasks requiring answer verification, category judgment or fine-grained reasoning. In this paper, we propose UMER, a Unified Multimodal Embedding and Ranking framework for universal multimodal retrieval. UMER replaces item-wise reflection with PairAware Discriminative Reasoning, which compares query-candidate pairs to identify instruction-relevant matching and discrepancy evidence. UMER jointly learns contrastive embeddings for efficient global matching and discriminative ranking for explicit pairwise relevance judgment within a single MLLM. A complementary mutual distillation strategy further transfers reliable pairwise preferences between the embedding and ranking functions. On MMEB-V2, UMER-R scores 65.5/70.6 with Qwen2-VL-2B/7B, outperforming the strongest included matched-backbone baselines by 1.4/2.0 points. UMER also supports budget-adjustable inference from efficient embedding retrieval to optional pair-aware reranking.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2608.18504 [cs.AI] |
| (or arXiv:2608.18504v3 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2608.18504 arXiv-issued DOI via DataCite |
Submission history
From: Xiyang Liu [view email]
[v1]
Wed, 19 Aug 2026 03:51:39 UTC (2,985 KB)
[v2]
Wed, 26 Aug 2026 06:46:11 UTC (2,985 KB)
[v3]
Wed, 11 Nov 2026 06:52:56 UTC (3,529 KB)
来源：arXiv cs.AI · arxiv.org