arXiv cs.AI· Zhinan Hou, Xingchen Li, Keyou You·· 9 天前AI 评分26
通过潜空间约束优化实现高效推理
Efficient Reasoning via Constrained Optimization in Latent Space
AI 导读
针对大推理模型（LRMs）过度思考导致 token 消耗过高的问题，研究者提出一种无需训练的框架：利用二次规划将偏离的隐藏状态投影回潜空间中高效推理步骤聚集的集中区域，从而在不牺牲性能的前提下减少 token 生成。
正文 · 原文
Abstract:Large Reasoning Models (LRMs) have shown remarkable reasoning capabilities, yet they still suffer from overthinking, generating redundant reasoning steps which incur substantial token consumption. Existing methods, such as suppressing reflective keywords or forcing shorter reasoning lengths, attempt to mitigate this issue but inevitably truncate necessary steps and induce underthinking, thereby compromising performance. To address this dilemma, we investigate the latent representations and observe that efficient reasoning steps naturally cluster into a concentrated region in latent space, while those deviating from this region tend to produce verbose sequences. To leverage this, we keep reasoning focused within this region via a quadratic program which projects deviating hidden states back into the region. Then we propose a novel training-free framework to achieve efficient reasoning that reduces token generation costs without sacrificing performance. Extensive experiments conducted on four models ranging from 1.5B to 14B, and across six benchmarks in math reasoning, coding, and scientific QA, validate the effectiveness of our method, up to a 12.1\% improvement in accuracy while reducing generated tokens by 11.8\% to 52.8\%. Codes are available at \href{this https URL}{this https URL}.
| Comments: | Accepted by NeurIPS2026 |
| Subjects: | Artificial Intelligence (cs.AI); Optimization and Control (math.OC) |
| Cite as: | arXiv:2609.34181 [cs.AI] |
| (or arXiv:2609.34181v2 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2609.34181 arXiv-issued DOI via DataCite |
Submission history
From: Zhinan Hou [view email]
[v1]
Mon, 28 Sep 2026 02:59:01 UTC (10,569 KB)
[v2]
Thu, 8 Oct 2026 16:41:45 UTC (10,569 KB)
来源：arXiv cs.AI · arxiv.org