arXiv cs.CL· Kristina \v{S}ekrst, Barbara Geld·· 12 小时前AI 评分31
研究指出 Natural Language Autoencoders 只能识别激活、无法解释其含义
Natural Language Autoencoders Identify Without Explaining
AI 导读
针对 Natural Language Autoencoders（NLAs）以“解释能否重建原激活”作为成功标准，该研究指出这一标准只检验信息是否充分，并不检验描述是否忠实：一句话即使错误解释了激活的作用，也可能足以识别该激活。
正文 · 原文
Abstract:Natural Language Autoencoders (NLAs) express a model's internal activations in the form of readable text. The verbalizer turns an activation into an English explanation; the reconstructor then converts that explanation back into an activation, and the explanation is considered successful if the round trip brings back the original activation. We claim that this criterion evaluates whether the information is sufficient and does not address descriptive faithfulness. A sentence can contain sufficient information to make it possible to identify an activation even if it wrongly explains what the activation does: in the case of reconstruction, the primary aim is to accurately identify the activations and not to describe them. This issue has the same structure as the chain-of-thought faithfulness problem, in which verbal output that has been trained to go with a process is taken as a report on that process, and it parallels the dissociation between an individual's introspective report and the actual governing process as documented in human psychology. Although proposed as interpretability tools for the purpose of alignment, NLAs reintroduce, through unverified explanations, the epistemic problem that they were intended to resolve.
| Comments: | ISPSM 2026 |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.32691 [cs.CL] |
| (or arXiv:2610.32691v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.32691 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Kristina Šekrst [view email]
[v1]
Fri, 23 Oct 2026 14:20:29 UTC (87 KB)
来源：arXiv cs.CL · arxiv.org