arXiv cs.AI· Youmin Ko, Hyemin Yang, Taeuk Kim, Hyunjoon Kim·· 3 天前AI 评分22
SATKGC：面向知识图谱补全的子图感知语言模型训练框架
Subgraph-Aware Training of Language Models for Knowledge Graph Completion Using Structure-Aware Contrastive Learning
AI 导读
研究者提出子图感知训练框架 SATKGC，通过子图感知 mini-batching 鼓励难负样本采样并缓解实体出现频率不平衡，同时引入新的对比学习机制，让模型更关注结构属性上更难的批内负三元组和正三元组。该工作称首次将知识图谱的结构归纳偏置全面融入 PLM 微调，在三个 KGC 基准上验证了优越性，代码已开源。
正文 · 原文
Abstract:Fine-tuning pre-trained language models (PLMs) has recently shown a potential to improve knowledge graph completion (KGC). However, most PLM-based methods focus solely on encoding textual information, neglecting the long-tailed nature of knowledge graphs and their various topological structures, e.g., subgraphs, shortest paths, and degrees. We claim that this is a major obstacle to achieving higher accuracy of PLMs for KGC. To this end, we propose a Subgraph-Aware Training framework for KGC (SATKGC) with two ideas: (i) subgraph-aware mini-batching to encourage hard negative sampling and to mitigate an imbalance in the frequency of entity occurrences during training, and (ii) new contrastive learning to focus more on harder in-batch negative triples and harder positive triples in terms of the structural properties of the knowledge graph. To the best of our knowledge, this is the first study to comprehensively incorporate the structural inductive bias of the knowledge graph into fine-tuning PLMs. Extensive experiments on three KGC benchmarks demonstrate the superiority of SATKGC. Our code is available.
| Comments: | Accepted to The Web Conference 2025 |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2407.12703 [cs.CL] |
| (or arXiv:2407.12703v5 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2407.12703 arXiv-issued DOI via DataCite |
Submission history
From: Youmin Ko [view email]
[v1]
Wed, 17 Jul 2024 16:25:37 UTC (8,250 KB)
[v2]
Fri, 19 Jul 2024 00:34:13 UTC (8,250 KB)
[v3]
Tue, 23 Jul 2024 06:26:30 UTC (8,250 KB)
[v4]
Tue, 15 Oct 2024 15:59:09 UTC (4,180 KB)
[v5]
Fri, 31 Jan 2025 12:56:16 UTC (1,626 KB)
来源：arXiv cs.AI · arxiv.org