arXiv cs.AI· Changguo Jia, Zhiyou Xiao, Xinbei Ma, Geng Chen, Weiming Zhang, Yiming Xu, Tianqi Zhao, Minghui Zhou·· 3 天前AI 评分39
PaceReasoner：无标签学习让长推理模型给出简短答案
PaceReasoner: Label-Free Learning to Answer Briefly from Long Reasoning
AI 导读
PaceReasoner 是一个无标签的测试时强化学习框架，用模型自身的长推理响应来提升短模式准确率。在三种模型骨干和四个基准上，其短模式平均准确率较基线提升 7.4–13.5 个百分点，平均每条响应少于 25 个推理 token。该方法通过渐进式置信度加权监督融合全响应与仅长响应的一致性，并用行为奖励与比例控制器平衡两种模式。
正文 · 原文
Abstract:Extended chain-of-thought reasoning improves problem-solving performance but increases inference cost and latency, motivating more accurate answers with shorter reasoning. We introduce PaceReasoner, a label-free test-time reinforcement learning framework that uses a model's LONG responses to improve its SHORT-mode accuracy. Both modes share a trainable policy, creating a need for reliable supervision, sufficient SHORT training samples, and continued LONG responses to guide learning. PaceReasoner combines all-response and LONG-only answer consensus through progressive, confidence-weighted supervision. A behavior reward gradually shifts from encouraging SHORT exploration to rewarding agreement with LONG answers, while a ratio controller regulates the balance between modes. Across three model backbones and four benchmarks, PaceReasoner improves average SHORT accuracy over the baselines by 7.4--13.5 percentage points, with fewer than 25 reasoning tokens per response on average. Ablations support the key components, and cross-dataset results demonstrate transfer beyond the adaptation set.
| Subjects: | Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.17900 [cs.AI] |
| (or arXiv:2610.17900v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.17900 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Changguo Jia [view email]
[v1]
Tue, 13 Oct 2026 07:55:23 UTC (555 KB)
来源：arXiv cs.AI · arxiv.org