arXiv cs.CL· Priyanka, Renaud Delbru, Souhail Bakkali, Micka\"el Coustaty, Antoine Doucet·· 13 小时前AI 评分31
近期大语言模型在标准基准上生成知识图谱到文本的实证研究
An Empirical Study of Recent Large Language Models for Knowledge Graph to Text Generation on Standard Benchmarks
AI 导读
研究对近期 LLM 与微调编码器-解码器模型在知识图谱到文本生成任务上进行基准对比，发现 LLM 在基于参考的指标上大幅缩小并在部分设置下追平差距，在基于图谱的指标上表现更优。基于 LLM 的评估分数与基于参考的指标相关性有限，说明两者捕捉的生成质量维度不同。基于图谱的评估显示，提示词驱动的模型比微调模型产生更少矛盾与三元组遗漏，且微调模型的这类错误随图谱规模增大而增长更快。
正文 · 原文
Abstract:Knowledge graph-to-text (KG-to-text) generation verbalizes a set of knowledge graph triples into fluent natural language. While fine-tuned encoder-decoder models have long been the standard approach, recent LLMs are becoming increasingly competitive. We benchmark recent LLMs against fine-tuned encoder-decoder models, using reference-based metrics and graph-grounded reference-free LLM-based evaluation validated against human annotations. We observed that recent LLMs substantially narrow, and in several settings close, the gap with fine-tuned models on reference-based metrics, while performing better on graph-grounded metrics. LLM-based evaluation scores show limited correlation with reference-based metrics, suggesting that the two approaches capture different aspects of generation quality. Graph-grounded evaluation reveals that prompted models produce fewer contradictions and triple omissions than fine-tuned models, and these errors increase more rapidly with graph size for fine-tuned models, highlighting differences between the two approaches that reference-based metrics may not fully capture.
| Comments: | 18 pages, 6 figures, 17 tables, Preprint, Under review |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.31529 [cs.CL] |
| (or arXiv:2610.31529v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.31529 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Priyanka - [view email]
[v1]
Thu, 22 Oct 2026 16:36:35 UTC (129 KB)
来源：arXiv cs.CL · arxiv.org