arXiv cs.CL· Yingyi Zhang, Chengzhi Zhang·· 11 小时前AI 评分27
利用大语言模型增强学术论文摘要与全文中的知识实体关系抽取
Enhancing Knowledge Entity Relation Extraction in Abstracts and Full-text Content of Academic Articles Using Large Language Models
AI 导读
研究系统评估了多种 LLM 策略在学术论文知识实体关系抽取（KERE）上的效果，并新增 SLM 增强开源 LLM、闭源 LLM 增强开源 LLM 两种策略。在自建的两个 NLP 领域标注数据集上，经 LoRA 适配的 T5 模型取得最高 macro F1，摘要与全文数据集分别为 0.7408 和 0.7013。
正文 · 原文
Abstract:This study investigates the effectiveness of large pre-trained language models (LLMs) for knowledge entity relation extraction (KERE) from academic papers. Existing studies have examined individual types of LLM-based strategies and evaluated model performance using metrics such as precision and recall. To provide a broader evaluation, this study investigates multiple strategies, including closed-source LLM-based CoT prompting, LoRA-adapted open-source LLMs, LLM-enhanced small pre-trained language models (SLMs), and SLM-enhanced closed-source LLMs. In addition, this study introduces two strategies, namely SLM-enhanced open-source LLMs and closed-source LLM-enhanced open-source LLMs. These strategies are evaluated from multiple perspectives, including metrics, their ability to mitigate errors observed in SLMs, and potential directions for model optimization. This study manually constructs two annotated datasets in the NLP domain, covering Abstract and Full-Text datasets. Experimental results show that the LoRA-adapted T5 model achieves the highest macro F1 scores of 0.7408 and 0.7013 on the Abstract and Full-Text datasets, respectively. SLMs also benefit from LLM-generated entity descriptions under the LLM-enhanced SLM strategy. Error analysis further reveals that the effectiveness of LLM-based strategies in mitigating SLM errors varies across error types. Based on the identified limitations and error patterns of these strategies, several optimization directions are proposed. In addition, combining strategies with different error patterns further improves KERE performance.
| Subjects: | Computation and Language (cs.CL); Digital Libraries (cs.DL); Information Retrieval (cs.IR) |
| Cite as: | arXiv:2610.41552 [cs.CL] |
| (or arXiv:2610.41552v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.41552 arXiv-issued DOI via DataCite (pending registration) |
|
| Journal reference: | Scientometrics, 2026 |
Submission history
From: Chengzhi Zhang [view email]
[v1]
Fri, 30 Oct 2026 10:50:28 UTC (9,050 KB)
来源：arXiv cs.CL · arxiv.org