arXiv cs.AI· Jie Peng, Yongxue Shan, Yongfu Zha, Cui Miao, Xiaodong Wang·· 10 小时前AI 评分20
NaviGATE:面向多模态知识图谱推理的噪声感知双视图生成框架
NaviGATE: Noise-Aware Dual-View Generation for Multimodal Knowledge Graph Reasoning
AI 导读
NaviGATE 是一个面向多模态知识图谱推理的噪声感知生成框架,先对齐图结构表示与文本语义,再用多模态实体摘要和结构 token 微调多模态语言模型完成结构感知的内部知识激活。
正文 · 原文
Abstract:Multimodal knowledge graph reasoning (MKGR) aims to infer missing facts by jointly exploiting relational structures, textual descriptions, and visual evidence. Existing embedding-based methods effectively capture graph topology, but their fixed multimodal representations are less effective for query-specific fine-grained semantic reasoning, while generation-based methods are vulnerable to noisy retrieval and may produce invalid entities. To address these challenges, we propose NaviGATE, a noise-aware generative framework for MKGR. NaviGATE first aligns graph-derived structural representations with textual semantics, then performs structure-aware internal knowledge activation by fine-tuning the multimodal language model with multimodal entity summaries and injected structural tokens. In the noise-aware dual-view instruction tuning stage, NaviGATE jointly optimizes retrieval-free and retrieval-aware views: reliable retrieved candidates are reinforced through contrastive logit sharpening, while misleading candidates are suppressed through parametric consistency distillation. In inference, structure-constrained decoding restricts generation to valid entities in the target knowledge graph. Experiments on standard benchmarks show that NaviGATE consistently outperforms strong unimodal, multimodal, and generation-based baselines, and remains robust under noisy retrieval scenarios.
| Comments: | 9 pages, 4 figures |
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.24552 [cs.AI] |
| (or arXiv:2611.24552v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2611.24552 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Jie Peng [view email]
[v1]
Wed, 18 Nov 2026 14:53:42 UTC (552 KB)
来源:arXiv cs.AI · arxiv.org