arXiv cs.AI· Shrestha Datta, Hongfu Liu, Anshuman Chhabra·· 11 小时前AI 评分29
黄金层在哪里：通过层梯度分析改进大语言模型的知识编辑
Golden Layers and Where to Find Them: Improved Knowledge Editing for Large Language Models Via Layer Gradient Analysis
AI 导读
研究提出 Layer Gradient Analysis（LGA）方法，通过梯度归因高效估计知识编辑的“黄金层”，避免在多次编辑运行中反复试错。作者假设存在固定黄金层可达到接近逐样本最优层的编辑效果，并用代理数据集识别这些层、验证其可泛化到未见测试集查询。在多个基准数据集上的实验显示，LGA 在不同 LLM 类型和多种知识编辑方法下均表现有效且稳健，该工作已被 NeurIPS 2026 主会接收。
正文 · 原文
Abstract:Knowledge editing in Large Language Models (LLMs) aims to update the model's prediction for a specific query to a desired target while preserving its behavior on all other inputs. This process typically involves two stages: identifying the layer to edit and performing the parameter update. Intuitively, different queries may localize knowledge at different depths of the model, resulting in different sample-wise editing performance for a fixed editing layer. In this work, we hypothesize the existence of fixed golden layers that can achieve near-optimal editing performance similar to sample-wise optimal layers. To validate this hypothesis, we provide empirical evidence by comparing golden layers against ground-truth sample-wise optimal layers. Furthermore, we show that golden layers can be reliably identified using a proxy dataset and generalize effectively to unseen test set queries across datasets. Finally, we propose a novel method, namely Layer Gradient Analysis (LGA) that estimates golden layers efficiently via gradient-attribution, avoiding extensive trial-and-error across multiple editing runs. Extensive experiments on several benchmark datasets demonstrate the effectiveness and robustness of our LGA approach across different LLM types and various knowledge editing methods.
| Comments: | Accepted to NeurIPS 2026 Main Conference |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2602.20207 [cs.LG] |
| (or arXiv:2602.20207v4 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2602.20207 arXiv-issued DOI via DataCite |
Submission history
From: Anshuman Chhabra [view email]
[v1]
Sun, 22 Feb 2026 22:55:11 UTC (4,290 KB)
[v2]
Fri, 27 Mar 2026 00:35:29 UTC (4,269 KB)
[v3]
Thu, 14 May 2026 19:27:02 UTC (6,768 KB)
[v4]
Wed, 4 Nov 2026 15:20:35 UTC (1,814 KB)
来源：arXiv cs.AI · arxiv.org