arXiv cs.CL· Usman Anwar, Tim Bakker, Dana Kianfar, Cristina Pinneri, Christos Louizos·· 13 小时前AI 评分38
用信息论分析与提升思维链可监控性
Analyzing and Improving Chain-of-Thought Monitorability Through Information Theory
AI 导读
研究用信息论分析指出，CoT 与输出之间的非零互信息是 CoT 可监控性的必要非充分条件，并提出信息差与诱导误差两个近似误差来源。实验显示，即使没有针对 CoT 的优化压力，CoT 混淆也可能出现；作者提出一种基于输出与 CoT 互信息的无标签训练目标，在多个环境中可防止 CoT 混淆，并在任务奖励设定不完美时缓解奖励黑客。
正文 · 原文
Abstract:Chain-of-thought (CoT) monitors are LLM-based systems that analyze reasoning traces to detect when outputs may exhibit attributes of interest, such as test-hacking behavior during code generation. In this paper, we use information-theoretic analysis to show that non-zero mutual information between CoT and output is a necessary but not sufficient condition for CoT monitorability. We identify two sources of approximation error that may undermine the performance of CoT monitors in practice: information gap, which measures the extent to which the monitor can extract the information available in CoT, and elicitation error, which measures the extent to which the monitor approximates the optimal monitoring function. Empirically, we show that CoT obfuscation can arise even in the absence of any optimization pressure against CoT. We further demonstrate that the fragility of CoT monitorability can be systematically mitigated through targeted training objectives. To this end, we propose a practical, label-free objective based on mutual information between outputs and CoTs, and benchmark it against an oracle-based method that directly rewards the monitored model for CoTs that maximize monitor accuracy. Across multiple environments, we show that our objective is effective at preventing CoT obfuscation even when training against a monitor, and helps mitigate reward hacking when the task reward is imperfectly specified.
| Comments: | The first two authors contributed equally |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Information Theory (cs.IT) |
| Cite as: | arXiv:2602.18297 [cs.LG] |
| (or arXiv:2602.18297v2 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2602.18297 arXiv-issued DOI via DataCite |
Submission history
From: Tim Bakker [view email]
[v1]
Fri, 20 Feb 2026 15:50:30 UTC (36,073 KB)
[v2]
Wed, 21 Oct 2026 13:35:19 UTC (20,032 KB)
来源：arXiv cs.CL · arxiv.org