arXiv cs.AI· Elias Sandmann, Sebastian Lapuschkin, Wojciech Samek·· 14 小时前AI 评分48
Leela Chess Zero 研究发现：学到的先验会覆盖神经网络中的前瞻计算
The Algorithm Is Not the Behavior: Learned Priors Override Look-Ahead in a Chess-Playing Neural Network
AI 导读
研究者在 Leela Chess Zero 的走子策略网络中发现“被遗忘的谜题”现象：正确解法（含一步将杀）常出现在中间层，却在最终输出中被系统性覆盖。复制此前分析显示前瞻机制本身正常运作，正确后续走法仍被表征且具因果作用；通过反向引导模型偏好，可恢复 71.7% 的被遗忘谜题，表明安全偏好压过了算法计算出的解。
正文 · 原文
Abstract:Recent mechanistic work has uncovered learned algorithms within neural networks, from modular arithmetic to search and planning in game-playing agents. But does algorithmic structure guarantee algorithmic behavior? We investigate this in Leela Chess Zero, the strongest neural chess engine, where prior work identified learned look-ahead. By extending the logit lens to its move-selecting policy network, we discover that correct puzzle solutions--including immediate checkmates--often appear in intermediate layers but are systematically overridden in the final output, a phenomenon we term "forgotten puzzles". Replicating prior analyses on these positions, we find that look-ahead operates normally--future moves of the correct continuation are represented, causally important, and linearly decodable--ruling out a failure of the algorithm. Instead, late layers increasingly shift toward safe play over aggression. To test whether this shift drives the override, we steer the model against these preferences and recover 71.7% of forgotten puzzles, providing causal evidence that safety priors override algorithmically computed solutions. These findings show that algorithmic structure does not guarantee algorithmic behavior: a model can internally solve a problem and still output the wrong answer.
| Comments: | NeurIPS 2026 (oral) |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2508.21380 [cs.LG] |
| (or arXiv:2508.21380v4 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2508.21380 arXiv-issued DOI via DataCite |
Submission history
From: Elias Sandmann [view email]
[v1]
Fri, 29 Aug 2025 07:51:45 UTC (4,230 KB)
[v2]
Tue, 25 Nov 2025 13:55:48 UTC (13,555 KB)
[v3]
Wed, 10 Jun 2026 10:31:06 UTC (38,855 KB)
[v4]
Thu, 29 Oct 2026 12:22:54 UTC (37,512 KB)
来源：arXiv cs.AI · arxiv.org