arXiv cs.CL· Jacob Solinsky (University of Minnesota), Serguei Pakhomov (University of Minnesota)·· 12 小时前AI 评分24
大语言模型如何编码与传播语法数：基于 SAE 的 Pythia-70M 研究
Encoding and Propagation of Grammatical Number in Large Language Models
AI 导读
研究者用稀疏自编码器（SAE）在 Pythia-70M 上定位到“复数名词”“复数主语”“生成复数动词”等特征，移除这些特征会让模型把“the men in the park”后的 are/do/have 改成单数形式，反向干预则相反。
正文 · 原文
Abstract:Sparse autoencoders (SAEs) have become an increasingly popular means of translating Large Language Model (LLM) hidden state vectors into a human-comprehensible list of features or symbols. As such, SAEs may provide insight into an LLM's knowledge representation and internal ``mental'' grammar - topics of pragmatic interest for mechanistic interpretation of AI behavior and of theoretical interest for the study of in silico representations of human language. Language models with fewer than 100M parameters, trained on small corpora, nonetheless exhibit strong grammatical competence and discourse coherence, despite producing content of limited reliability. We use SAEs that we trained on one such model (Pythia-70M) to conduct an in-depth examination of how a single grammatical phenomenon, number agreement, is represented and processed in an LLM. We demonstrate the mechanism and the methodology by which SAEs can be used to find and propagate ``plural noun'', ``plural matrix subject'', and ``generate a plural verb'' features which cause the LLM to produce plural verb forms such as ``are'', ``do'', or ``have'' instead of ``is'', ``does'', and ``has'' in response to prompts such as ``the men in the park''. We show that intervening to remove these features from the hidden states of plural subject noun tokens causes the model to produce singular verbs instead, and vice versa. We also show that corresponding ``singular noun'' and ``generate a singular verb'' features are not as effective for this task. This asymmetry suggests that Pythia-70m makes use of a privative representation of grammatical number in English. These findings replicated with larger LLMs Gemma-2-2b and Llama-3.1-8b yielded similar results. Our main contribution is a novel approach to the use of SAEs to understanding how LLMs encode human language that addresses a number of technical and methodological challenges.
| Comments: | 50 pages, 19 figures |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.07283 [cs.CL] |
| (or arXiv:2611.07283v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2611.07283 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Jacob Solinsky [view email]
[v1]
Thu, 5 Nov 2026 01:04:02 UTC (2,596 KB)
来源：arXiv cs.CL · arxiv.org