arXiv cs.CL· Alberto Cattaneo, Carlo Luschi, Daniel Justus·· 3 天前AI 评分22
SynthKGQA：为知识图谱增强 LLM 生成真值子图以改进训练与评测
Ground-Truth Subgraphs for Better Training and Evaluation of Knowledge Graph Augmented LLMs
AI 导读
研究者提出 SynthKGQA，一个由 LLM 驱动的框架，可从任意知识图谱生成高质量知识图谱问答数据集，并提供完整的真值事实集合。基于 Wikidata 衍生的 ogbl-wikikg2 构建的 GTSQA 数据集，专门测试对未见图结构和关系类型的零样本泛化能力。实验显示，在 GTSQA 上训练的 KG 检索器可迁移到人工标注基准，且其真值子图比以往启发式方法提供更好的训练监督信号。
正文 · 原文
Abstract:Retrieval of information from graph-structured knowledge bases represents a promising direction for improving the factuality of LLMs. While various solutions have been proposed, a comparison of methods is difficult due to the lack of challenging QA datasets with ground-truth targets for graph retrieval. We present SynthKGQA, an LLM-powered framework for generating high-quality Knowledge Graph Question Answering datasets from any Knowledge Graph, providing the full set of ground-truth facts in the KG to reason over questions. To demonstrate its utility, we apply SynthKGQA to ogbl-wikikg2, a Wikidata-derived KG, and validate generated items against an up-to-date dump of Wikidata to create GTSQA. This new dataset is specifically designed to test zero-shot generalization with respect to unseen graph structures and relation types, enabling us to analyze the abilities and limitations of SOTA graph retrieval approaches at an unprecedented level of granularity. We also show that KG retrievers trained on GTSQA can transfer to a human-curated benchmark, and that the ground-truth subgraphs produced by SynthKGQA provide a better training supervision signal than previously-used heuristics.
| Comments: | Published in Transactions on Machine Learning Research (TMLR) |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Information Retrieval (cs.IR) |
| Cite as: | arXiv:2511.04473 [cs.LG] |
| (or arXiv:2511.04473v4 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2511.04473 arXiv-issued DOI via DataCite |
|
| Journal reference: | Transactions on Machine Learning Research (TMLR), 09/2026 |
Submission history
From: Daniel Justus [view email]
[v1]
Thu, 6 Nov 2025 15:45:18 UTC (2,273 KB)
[v2]
Thu, 4 Dec 2025 08:34:50 UTC (2,504 KB)
[v3]
Mon, 21 Sep 2026 07:52:30 UTC (2,726 KB)
[v4]
Wed, 14 Oct 2026 08:53:00 UTC (2,726 KB)
来源：arXiv cs.CL · arxiv.org