arXiv cs.CL· Hengyu Jin, Shu Yang, Di Wang·· 12 小时前AI 评分31
仅看最终 checkpoint 不够：分析训练轨迹中潜在推理的忠实性
Final Checkpoints Are Not Enough: Analyzing Latent Reasoning Faithfulness Along Training Trajectories
AI 导读
研究追踪训练全程发现，潜在推理模型的准确率上升时，对改变推理路径的反事实编辑反而变得不敏感。在 ProsQA 图可达任务上，输出对噪声替换的敏感度与反事实响应性同步下降；GSM8K 的二元选择与开放式版本在干预所有潜在状态时呈现相反的敏感度轨迹。基于这些轨迹干预训练过程可提升反事实响应性，说明只评估最终 checkpoint 会掩盖潜在状态贡献的变化。
正文 · 原文
Abstract:Latent reasoning performs multi-step inference in continuous hidden states, promising more compact and efficient reasoning. However, these opaque states raise a question of faithfulness: whether the latent reasoning steps drive the final answer. Prior work reports unfaithful behaviors at selected checkpoints, leaving their development during training unexamined. We track behavioral and activation-based evidence across training using verified counterfactual edits and interventions on the latent reasoning states. We find that accuracy can improve while models become less responsive to edits that change the required reasoning path and the correct answer. This counterfactual responsiveness follows distinct trajectories across latent reasoning approaches. On the graph reachability task ProsQA, output sensitivity to noise replacement declines alongside counterfactual responsiveness. Across separately trained binary-choice and open-ended versions of GSM8K math problems, interventions on all latent states show opposite sensitivity trajectories. Guided by these trajectories, we intervene on the training procedure and improve counterfactual responsiveness. These results show that evaluating only a final checkpoint can obscure both when counterfactual responsiveness changes and what the latent states contribute.
| Subjects: | Machine Learning (cs.LG); Computation and Language (cs.CL) |
| Cite as: | arXiv:2607.06648 [cs.LG] |
| (or arXiv:2607.06648v3 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2607.06648 arXiv-issued DOI via DataCite |
Submission history
From: Hengyu Jin [view email]
[v1]
Tue, 7 Jul 2026 16:09:44 UTC (297 KB)
[v2]
Tue, 1 Sep 2026 03:23:56 UTC (284 KB)
[v3]
Wed, 4 Nov 2026 08:13:40 UTC (361 KB)
来源：arXiv cs.CL · arxiv.org