arXiv cs.CL· William Dai, Zhenghao Herbert Zhou, Maya Viswanathan, Robert Frank·· 12 小时前AI 评分20
用影响函数识别语法泛化中的间接证据
Where's the Indirect Evidence? Investigating Grammatical Generalization with Influence Functions
AI 导读
研究用影响函数（Koh and Liang, 2017）衡量单条训练句对模型预测的影响，覆盖 BLiMP 的 15 种语法现象，并用人工规则过滤掉直接证据，取剩余影响力最高的 60,000 句作为间接证据。
正文 · 原文
Abstract:Language models generalize beyond their training corpus to learn linguistic phenomena (Patil et al., 2024; Misra and Mahowald, 2024), showing that there exists indirect evidence driving learning. Prior work has inferred that indirect evidence exists without identifying its character or considered specific hypotheses about what it might be. We explore an alternative bottom-up method for identifying indirect evidence. We use influence functions (Koh and Liang, 2017) to measure how individual training sentences affect model predictions in 15 grammatical phenomena from BLiMP (Warstadt et al., 2020). We filter out a phenomenon's direct evidence using hand-constructed rules and take the 60,000 most influential remaining sentences as indirect evidence. This indirect evidence substantially affects learning: retraining without it lowers held-out accuracy more than removing a matched random group for every phenomenon across five models (by a median of 12 points). We characterize the indirect evidence by its similarity to our direct evidence filters: it (partially) meets the filters' structural conditions at a median of 4.7 times the rate of a matched random group. Finally, clustering phenomena by indirect-evidence overlap recovers three of the five BLiMP categories that include multiple phenomena, suggesting that this method can reveal linguistic relationships.
| Comments: | 18 pages |
| Subjects: | Computation and Language (cs.CL); Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.22919 [cs.CL] |
| (or arXiv:2610.22919v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.22919 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: William Dai [view email]
[v1]
Fri, 16 Oct 2026 04:32:36 UTC (181 KB)
来源：arXiv cs.CL · arxiv.org