跳到正文
arXiv cs.AI· Jie Peng, Yongxue Shan, Yongfu Zha, Xiaodong Wang·· 10 小时前AI 评分18

MECI:面向多模态实体链接的多元素协同交互框架

MECI: Multi-Element Collaborative Interaction for Multimodal Entity Linking

AI 导读

研究提出多模态实体链接框架 MECI,通过多视图专家网络与 KL 引导路由机制解耦特征学习,缓解元素主导问题。其分层多模态交互模块用动态模态感知加权整合多粒度证据,并引入大语言模型做零样本重排序。在 WikiDiverse、RichpediaMEL 和 WikiMEL 上,Hits@1 分别提升 1.95%、7.30% 和 2.31%。

正文 · 原文

View PDF HTML (experimental)

Abstract:Multimodal Entity Linking (MEL) aims to disambiguate mentions in multimodal contexts by grounding them to specific entities in a knowledge base. A pivotal challenge in MEL is capturing multi-level correspondences: the semantic consistency between mention-entity pairs and the complementary correlations across modalities. However, existing methods often suffer from element dominance due to their reliance on coupled interactions or coarse global aggregations. In response, we propose the Multi-Element Collaborative Interaction (MECI) framework. First, to capture multi-element mention-entity correspondences, we develop a Multi-view Experts Network that leverages a "divide-and-conquer" strategy for decoupled feature learning to mitigate element dominance, supported by a KL-guided routing mechanism that governs expert specialization and collaboration. Furthermore, to model cross-modal complementary correlations, we propose a Hierarchical Multimodal Interaction Module, where a dynamic modality-aware weighting network refines interactions across hierarchical semantic levels, thereby integrating multi-granular evidence to counteract element dominance. Finally, we incorporate a generative semantic refinement stage that utilizes large language models for zero-shot re-ranking. Extensive experiments on WikiDiverse, RichpediaMEL, and WikiMEL show that MECI consistently outperforms state-of-the-art baselines, improving Hits@1 by 1.95%, 7.30%, and 2.31%, respectively.
Comments: 11 pages, 8 figures
Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
Cite as: arXiv:2611.24522 [cs.CL]
  (or arXiv:2611.24522v1 [cs.CL] for this version)
  https://doi.org/10.48550/arXiv.2611.24522

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Jie Peng [view email]
[v1] Wed, 18 Nov 2026 14:39:07 UTC (5,275 KB)

来源:arXiv cs.AI · arxiv.org