arXiv cs.CL· Linyu Li, Zhi Jin, Yuanpeng He, Dongming Jin, Haoran Duan, Zhengwei Tao, Xuan Zhang, Jiandong Li, Huanyao Zhang, Gadeng Luosang, Nyima Tashi·· 12 小时前AI 评分22
重新审视知识图谱嵌入的泛化能力：实证研究与稀疏峰值正则化
Rethinking Generalization in Knowledge Graph Embedding: An Empirical Study and Sparse Peak Regularization
AI 导读
一项覆盖翻译模型、张量分解、GNN、时序与多模态等知识图谱嵌入（KGE）模型的实证研究发现，过拟合在各模型家族中普遍存在，适当正则化可显著提升泛化与预测性能。基于此提出的稀疏峰值正则化（SPR）将基于排序的选择性稀疏引入正则化目标，只惩罚主导高能量嵌入分量、不惩罚低能量分量。在八个基准数据集上，SPR 在几乎所有设置下均优于 N3、DURA、ER 等现有正则化方法。
正文 · 原文
Authors:Linyu Li, Zhi Jin, Yuanpeng He, Dongming Jin, Haoran Duan, Zhengwei Tao, Xuan Zhang, Jiandong Li, Huanyao Zhang, Gadeng Luosang, Nyima Tashi
Abstract:Knowledge graphs (KGs) provide structured representations of entities and relations, and underpin data management and Web applications such as search, recommendation, and question answering. Since Web-scale KGs such as Freebase and Wikidata are inherently incomplete, knowledge graph embedding (KGE) maps entities and relations into continuous vector spaces and is widely used to infer missing facts through link prediction. Despite extensive progress in KGE model design, the role of regularization has been less systematically examined, especially beyond tensor-decomposition models. The primary contribution of this paper is a comprehensive empirical study of regularization for KGE from the perspective of generalization. Across translation-based, tensor-decomposition-based, GNN-based, temporal, and multimodal KGE models, we show that overfitting is common across model families and that appropriate regularization can substantially improve both generalization and predictive performance. Motivated by these findings, we further propose Sparse Peak Regularization (SPR), a simple and model-agnostic regularizer that introduces rank-based selective sparsity into the regularization target. SPR concentrates the penalty on dominant high-energy embedding components, which contribute most to the embedding norm and overfitting risk, while leaving low-energy components unpenalized so that weak but useful signals are not unnecessarily suppressed. Experiments on eight benchmark datasets and diverse KGE models show that SPR achieves the best results in nearly all settings compared with existing regularizers such as N3, DURA, and ER. These results show that regularization deserves more systematic attention in KGE, and that selective regularization is an effective and broadly applicable strategy for improving the generalization of KGE models.
| Subjects: | Machine Learning (cs.LG); Computation and Language (cs.CL) |
| Cite as: | arXiv:2505.23442 [cs.LG] |
| (or arXiv:2505.23442v2 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2505.23442 arXiv-issued DOI via DataCite |
Submission history
From: Linyu Li [view email]
[v1]
Thu, 29 May 2025 13:39:18 UTC (3,579 KB)
[v2]
Tue, 20 Oct 2026 09:19:21 UTC (952 KB)
来源：arXiv cs.CL · arxiv.org