arXiv cs.CL· Weijiang Lv, Wentong Zhao, Jiayu Wang, Yuhao Wu, Jiaheng Wei, Xiaobo Xia·· 14 小时前AI 评分34
GeoFaith：用时空潜在动力学让大语言模型学习忠实的推理过程
Learning Faithful Reasoning Processes with Spatio-Temporal Latent Dynamics for Large Language Models
AI 导读
GeoFaith 提出一种时空潜在动力学框架，无需过程奖励模型即可让 LLM 学习忠实的推理过程。该方法将最终状态的潜在不确定性与预测熵动力学作为过程正则项融入 GRPO，联合优化答案正确性与推理质量，无需步骤级过程标注。实验显示其推理准确率持续提升、推理轨迹更高效，优于仅用结果奖励的强化学习和过程监督基线。
正文 · 原文
Abstract:Chain-of-Thought (CoT) reasoning has significantly enhanced the reasoning capabilities of large language models (LLMs). However, existing reinforcement learning methods primarily optimize outcome-level rewards, providing limited supervision over the reasoning process. As a result, models may produce shortcut-prone or unstable reasoning trajectories even when the final answer is correct. We propose GeoFaith, a spatio-temporal latent dynamics framework for learning faithful reasoning processes without process reward models. GeoFaith combines final-state latent uncertainty, which measures whether a completed trajectory lies in a well-supported manifold region, with predictive-entropy dynamics, which penalize abnormal uncertainty patterns such as premature collapse, oscillation, and prolonged uncertainty. These signals are incorporated as process regularization into GRPO, jointly optimizing answer correctness and reasoning quality without step-level process annotations. Experiments show that GeoFaith consistently improves reasoning accuracy, produces more efficient reasoning trajectories, and outperforms both outcome-only reinforcement learning and process-supervision baselines.
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2605.26893 [cs.CL] |
| (or arXiv:2605.26893v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2605.26893 arXiv-issued DOI via DataCite |
Submission history
From: Xiaobo Xia [view email]
[v1]
Tue, 26 May 2026 11:54:40 UTC (10,648 KB)
[v2]
Mon, 26 Oct 2026 12:02:16 UTC (6,864 KB)
来源：arXiv cs.CL · arxiv.org