arXiv cs.CL· Kevin Yandoka Denamgana\"i·· 12 小时前AI 评分37
自然语言自编码器（NLA）的可监控性研究：识别与判别之别
On the Monitorability of Natural Language Autoencoders: Identification vs Discrimination
AI 导读
研究将自然语言自编码器（NLA）的训练与监控部署形式化为两种不同的指称博弈，指出训练从不要求解释排除错误选项，因此 Activation Verbaliser（AV）的解释只能识别所描述的激活，却难以判别真伪。
正文 · 原文
Abstract:Natural Language Autoencoders (NLAs) are proposed as oversight tools for language models: an Activation Verbaliser (AV) writes an English explanation of an activation vector, and an Activation Reconstructor (AR) maps the explanation back to a vector. Their value for oversight depends on a human monitor being able to tell a true account of an activation from a false one, given the AV's explanation. We formalise NLA training and NLA oversight deployment as two different referential games from Emergent Communication: training is a generative referential game, while oversight is a discriminative, multi-modal, partially-observable referential game played over a many-to-many relation between the monitored language model's activations and the accounts that are true of them. The gap between the two games predicts that AV's explanations will identify which activation they describe without discriminating its true account from closely matched false ones, since training never asks explanations to rule anything out. Using the NLA's own AR as a maximally-sympathetic surrogate monitor model, we experimentally confirm this theoretical insight on one released checkpoint: true accounts beat accounts of other activations in all 320 comparisons, but beat fluent rivals differing in a single claim only 60.2% of the time (standard error 6.7%, chance at 50%, $n = 35$). A meaning-preserving rewrite by an uninvolved model costs only 2.1% of the distance to another activation's account, so the failure is not due to the NLA pair having developed private conventions in the AV's explanations. Our rival accounts test only transcript-level truths, which the monitored language model's conversation settles, so these results bound the easiest case and leave claims about the monitored model's own processing untested. We state conjectures that would make both results consequences of the framework.
| Comments: | work in progress, open draft |
| Subjects: | Machine Learning (cs.LG); Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.18727 [cs.LG] |
| (or arXiv:2610.18727v2 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.18727 arXiv-issued DOI via DataCite |
Submission history
From: Kevin Denamganaï [view email]
[v1]
Tue, 13 Oct 2026 17:05:28 UTC (1,505 KB)
[v2]
Tue, 20 Oct 2026 17:52:33 UTC (1,503 KB)
来源：arXiv cs.CL · arxiv.org