跳到正文
arXiv cs.CL· Igor Shilov, Thomas Fel, Tal Haklay, Lucius Bushnaq, Owen Lewis·· 23 小时前AI 评分34

大规模神经元社区发现揭示语言模型中的神经元稀疏行为

Large-scale Neuron Community Discovery Reveals Neuron-Sparse Behaviors in Language Models

AI 导读

研究者提出神经元社区发现方法,结合梯度归因、神经元共现图与社区检测,在 Llama-3.1-8B 上用一千万条提示词发现数千个可复用的神经元社区。这些社区经必要性、层内充分性和层必要性三项因果测试验证,支持语法一致、token 概率校准、固定短语、上下文复制、加法与乘法等功能。

正文 · 原文

View PDF HTML (experimental)

Abstract:What reusable units of computation can we discover within a language model's neurons? Prior work has shown that individual forward passes can often be sparsified post-hoc to small groups of neurons, but it has been unclear whether these groups form coherent computational entities reused across inputs. We introduce neuron community discovery, an unsupervised method for finding neuron groups that recur across prompts and are causally necessary and sufficient, within the layers they span, for common computations. The method combines gradient-based attribution, a graph connecting neurons that are repeatedly attributed together, and community detection. Applied to Llama-3.1-8B using ten million prompts, it discovers thousands of candidate communities. We evaluate them on held-out prompts through three causal tests: community necessity, sufficiency within a layer subset, and layer necessity. These tests identify groups whose removal disrupts predictions and whose retention preserves predictions despite ablating the remaining MLP neurons in the tested layers. Discovered communities support functions like grammatical agreement, token-probability calibration, fixed phrases, in-context copying, addition, and multiplication. With communities capped at 128 neurons across all layers, 10.3% of held-out prompts have at least one layer's contribution explained on a neuron-sparse basis. Coverage remains 8.7% when requiring at least four layers and 3.3% when requiring at least twelve. These results show that small groups of existing neurons can provide reusable computational components across prompts, and that such groups can be discovered at scale.
Subjects: Machine Learning (cs.LG); Computation and Language (cs.CL)
Cite as: arXiv:2611.23112 [cs.LG]
  (or arXiv:2611.23112v1 [cs.LG] for this version)
  https://doi.org/10.48550/arXiv.2611.23112

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Igor Shilov [view email]
[v1] Tue, 17 Nov 2026 17:33:20 UTC (945 KB)

来源:arXiv cs.CL · arxiv.org