arXiv cs.CL· Alexander Boesgaard Lorup·· 12 小时前AI 评分39
Concise-9B：通过简洁自训练削减推理模型的多余思考
Concise-9B: Removing the Slack from a Reasoning Model's Thinking
AI 导读
研究者用两轮 on-policy 拒绝采样微调，保留最短正确样本并混入少量答案合成目标，将 Qwen3.5-9B 训练为 Concise-9B。在 8,983 道留出题上，32K 思考预算下其生成 token 数为基座的 0.66x，准确率差异在 ±1 点内（-0.20 点）。
正文 · 原文
Abstract:Reasoning models answer better when they think longer, but a thinking trace keeps going well after its answer has formed. We measure this slack in Qwen3.5-9B and remove most of it with concise self-training: two rounds of on-policy rejection-sampling fine-tuning that keep the shortest correct samples, plus a small share of answer-synthesis targets, giving Concise-9B. On 8,983 held-out problems (competition and hard mathematics, SuperGPQA, a five-area confirmation set and MMLU-Pro), Concise-9B generates 0.66x the tokens of Qwen3.5-9B at a 32K thinking budget, with accuracy equivalent within +/-1 point (-0.20 [-0.77, +0.36] points). Forcing each trace on the hard and confirmation sets to answer every 2,048 thinking tokens, we find that 54-64% of Qwen3.5-9B's thinking tokens come after the point from which every forced answer equals the final one (41-48% for Concise-9B). Stopping there would keep every answer at 0.35-0.44x the base model's tokens, but finding that point takes hindsight: stopping when three consecutive forced answers agree costs the base model 2.4-9.2 points. Concise self-training removes slack without that cost: 86% [83, 90] of the thinking tokens it saves come after the answer has formed. It has a limit: with traces continued to 80K tokens, Concise-9B solves 25 fewer of 1,276 hard problems (-25 [-49, -1]), mostly in competition mathematics. The measurement also yields a cheap commit signal for adaptive sampling that matched two-sample escalation within +/-1 point at 0.67x its tokens in a pre-registered test. Results on Qwen3.5-4B, on which the recipe was developed, and a replication on Olmo-3-7B-Think suggest that the saving depends on how verbose the base model is.
| Comments: | 18 pages, 5 figures, 14 tables |
| Subjects: | Machine Learning (cs.LG); Computation and Language (cs.CL) |
| Cite as: | arXiv:2611.02278 [cs.LG] |
| (or arXiv:2611.02278v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2611.02278 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Alexander Lorup [view email]
[v1]
Sun, 1 Nov 2026 23:02:43 UTC (208 KB)
来源：arXiv cs.CL · arxiv.org