arXiv cs.AI· Zongyu Wu, Yilong Wang, Xiaochen Wang, Minhua Lin, Zhichao Xu, Fenglong Ma, Xiang Zhang, Suhang Wang·· 4 小时前AI 评分24
CEMMKG:面向多模态知识图谱 RAG 的多粒度上下文增强框架
Multi-Granularity Context-Enhanced RAG over Multimodal Knowledge Graphs
AI 导读
研究者提出 CEMMKG 框架,通过为每张图像补充局部与全局文本上下文来构建上下文增强的多模态知识图谱,以缓解图文语义鸿沟。局部上下文除周边文本外还纳入与图像语义相关的句子,并采用多粒度设计捕捉不同细节层级的信息,全局上下文则提供整篇文档摘要。多数据集实验表明,该框架能有效利用上下文信息提升基于 MMKG 的 RAG 性能,且在不同 MMKG-based RAG 方法上均有效。
正文 · 原文
Abstract:Knowledge graph (KG)-based retrieval-augmented generation (RAG) leverages structured knowledge to provide large language models (LLMs) with high-quality external information. Building on these works, recent studies have explored multimodal knowledge graphs (MMKGs) as knowledge bases for GraphRAG. This enables GraphRAG to integrate knowledge across multiple modalities, thereby further enhancing its performance. However, existing MMKG-based RAG methods generally follow a common pipeline in which different modalities are largely processed independently before being fused. As a result, textual context is only used to a limited extent during visual information extraction and subsequent multimodal knowledge fusion. This brings a semantic gap between images and text which limits the multimodal GraphRAG performance. To address this issue, we propose a novel framework for constructing a Context-Enhanced MMKG (CEMMKG) to better support multimodal GraphRAG. The proposed CEMMKG enriches each image with complementary textual context at both local and global scopes. Local context goes beyond the surrounding text by incorporating sentences that are semantically related to the image, while global context provides a summary of the entire document. We further introduce a multi-granularity design for the local context, allowing it to capture semantically relevant information at different levels of detail. Extensive experiments on multiple datasets validate that CEMMKG is effective in leveraging contextual information to improve MMKG-based RAG performance. Moreover, its effectiveness across different MMKG-based RAG methods demonstrates its broad applicability.
| Comments: | Preprint |
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2608.25986 [cs.AI] |
| (or arXiv:2608.25986v2 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2608.25986 arXiv-issued DOI via DataCite |
Submission history
From: Zongyu Wu [view email]
[v1]
Wed, 26 Aug 2026 16:38:02 UTC (553 KB)
[v2]
Tue, 17 Nov 2026 11:26:05 UTC (311 KB)
来源:arXiv cs.AI · arxiv.org