arXiv cs.CL· Dimitri Kachler, Damien Sileo, Pascal Denis·· 12 小时前AI 评分26
Influcoder：将解码器梯度影响排名蒸馏进编码器，实现大规模数据归因
Influcoder: Distilling Decoders' Gradient Influence Rankings into an Encoder for Data Attribution
AI 导读
Influcoder 通过将解码器的梯度影响排名蒸馏进编码器，实现快速、低成本的大规模影响函数数据归因。现有影响函数方法虽有效，但处理速度和存储紧凑性不足，难以在大型数据集上实际部署。该论文已被 AACL-IJCNLP 2026 主会议接收。
正文 · 原文
Abstract:With the growth of LLMs' (Large Language Models) capabilities, there has been an increasing push to curate high quality datasets by filtering samples in the training data. In general, Data Attribution (DA) methods aim to estimate how individual samples in a training dataset can precondition a model to generate certain outputs. As an example, one might be interested in which samples in the data could be the source of toxic behavior after training the LLM. Many methods quantify this conditioning through the paradigm of influence functions. While methods of this family are effective in its function, they lack the necessary processing speed and storage compactness to be practically implemented on large datasets. We propose a method, Influcoder, as a quick and cost-effective approach to influence-based Data Attribution at scale.
| Comments: | Accepted to AACL-IJCNLP 2026 (Main Conference). 9 pages, 4 figures |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2606.13668 [cs.CL] |
| (or arXiv:2606.13668v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2606.13668 arXiv-issued DOI via DataCite |
|
| Journal reference: | Proceedings of IJCNLP-AACL 2026 (Volume 2: Short Papers), pages 780-788 |
Submission history
From: Dimitri Kachler [view email]
[v1]
Thu, 11 Jun 2026 17:58:33 UTC (366 KB)
[v2]
Thu, 5 Nov 2026 18:19:40 UTC (330 KB)
来源：arXiv cs.CL · arxiv.org