arXiv cs.CL· Piotr Wilam·· 11 小时前AI 评分31
校准概念画像：让模型神经元读取成为可测量的方法
Calibrated concept profiles: making the reading of a model's neurons a measurement
AI 导读
研究者提出"校准概念画像"方法，将直接读取模型神经元变为可重复的测量：以参考语料中各因素的出现率声明刺激分布，生成提示词电池并两次抽样，为每个神经元给出以自身在普通输入上的波动为单位的单一数值，两次抽样提供噪声单位。
正文 · 原文
Abstract:Interpretability has largely moved from reading a model's neurons directly to reading a second model trained on the first: a probe, a sparse autoencoder, a crosscoder. The move answered real faults -- mixed meanings, thresholds without a unit, readings that did not transfer from one study to the next -- but gave up what a direct reading alone provides: nothing trained in between, the neurons themselves named, and a reading that can be repeated and compared. Here the direct reading is made a measurement.
A neuron's activation varies with the prompt, so its reading is a distribution over a population of stimuli. We declare that population: every factor around a target construct is drawn at its rate in a reference corpus and logged.
From the declared distribution we generate a battery of prompts, and draw it twice. From the battery we compute the calibrated concept profile, one number per neuron, stated in units of the neuron's own spread on ordinary input; the two draws give every number a noise unit. Before the profile is read, a fixed set of checks licenses the battery on the model, and the smallest change the profile can resolve across training checkpoints is measured.
Realised for Python on OpenCoder-1.5B and followed across checkpoints spanning two trillion training tokens, the profile gives what the alternatives do not report: a unit on every number, a chance level for every comparison, and which neurons carry a construct at every checkpoint.
The battery, the code and the data behind every result are released.
| Comments: | 25 pages, 5 figures, 11 tables. Dataset: AtlasOpenCoder, tag v1, doi:https://doi.org/10.57967/hf/11472. Code: this https URL |
| Subjects: | Computation and Language (cs.CL); Machine Learning (cs.LG) |
| ACM classes: | I.2.6; I.2.7 |
| Cite as: | arXiv:2611.09196 [cs.CL] |
| (or arXiv:2611.09196v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2611.09196 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Piotr Wilam [view email]
[v1]
Fri, 6 Nov 2026 09:17:39 UTC (446 KB)
来源：arXiv cs.CL · arxiv.org