跳到正文
arXiv cs.AI· Sida Liu, Feijiang Han, Sarah Wiegreffe, Furong Huang·· 4 小时前AI 评分38

ICA Lens:非高斯性揭示语言模型中的可解释方向

ICA Lens: Non-Gaussianity Reveals Interpretable Directions in Language Models

AI 导读

研究者提出 ICA Lens,直接在 LLM 残差流激活空间中搜索非高斯方向来发现可解释特征,无需训练稀疏字典。在 GPT-2 Small、Gemma 2 2B 和 Qwen 3.5 9B Base 上,恢复的方向捕获了连贯的词汇、句法和语义模式,非高斯性与人类标签置信度及自动可解释性正相关。

正文 · 原文

View PDF HTML (experimental)

Abstract:Finding human-interpretable directions in language-model activations is important for understanding and controlling model behavior. These directions are usually discovered through indirect learning objectives such as sparse reconstruction, but it remains unclear what existing statistical structure distinguishes interpretable directions and whether that structure can be used directly for feature discovery. We find that non-Gaussianity provides such a signature: projection distributions of natural-text residual-stream activations along sparse autoencoder (SAE) decoder directions are substantially more non-Gaussian than those along random directions, with most contexts forming a concentrated background and recurring patterns often occupying one tail. This raises a natural question: if interpretable directions tend to have non-Gaussian projection distributions, can we discover them by searching for non-Gaussian directions directly in activation space, without training another sparse dictionary? To test this hypothesis, we introduce ICA Lens, a practical workflow for efficient discovery, reliable annotation, and faithful evaluation of non-Gaussian directions in LLM residual-stream activations. Across GPT-2 Small, Gemma 2 2B, and Qwen 3.5 9B Base, we find that (1) the recovered non-Gaussian directions capture coherent lexical, syntactic, and semantic patterns under both human and LLM-based evaluation; (2) non-Gaussianity is positively associated with both human-label confidence and automated interpretability; and (3) the resulting coordinates are competitive with SAEs across interpretability, sparse reconstruction, concept prediction, and causal interventions, while fitting in minutes per layer and using 7-43 times less memory. These results show that non-Gaussianity is a direct unsupervised signal for discovering interpretable and useful structure in activations.
Comments: Added experiments and refined the presentation
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
Cite as: arXiv:2606.11722 [cs.LG]
  (or arXiv:2606.11722v3 [cs.LG] for this version)
  https://doi.org/10.48550/arXiv.2606.11722

arXiv-issued DOI via DataCite

Submission history

From: Feijiang Han [view email]
[v1] Wed, 10 Jun 2026 06:53:58 UTC (2,296 KB)
[v2] Mon, 19 Oct 2026 17:23:26 UTC (6,395 KB)
[v3] Tue, 17 Nov 2026 04:23:38 UTC (7,693 KB)

来源:arXiv cs.AI · arxiv.org