arXiv cs.CL· Zhongyin Wang, Zhaoqian Xue, Junting Mu, Haozheng Liu, Ye Yuan, Haolun Wu, Lingyao Li·· 14 小时前AI 评分34
在预算约束下截断推理：将 System-1 模型作为决策者进行评估
Truncated Reasoning Under a Budget: Evaluating System-1 Models as Decision-Makers
AI 导读
研究将 Jev 作为截断推理的决策读取器，在七个 LLM 推理模型与五个多选题基准的 35 组组合上评估，在 500 至 16,000 token 预算下平均准确率均高于强制作答。
正文 · 原文
Abstract:Large language models (LLMs) often generate more reasoning tokens than needed. Limiting the reasoning budget reduces cost, but forcing an answer from an incomplete trace can hurt accuracy. System-1 decision models select among candidate options without producing a reasoning response. We evaluate Jev as a decision reader of truncated reasoning across seven LLM reasoners and five multiple-choice benchmarks (35 pairs). Jev achieves higher mean accuracy than forced answering at tested budgets from 500 to 16,000 tokens, including the estimated costs of reasoning and answering. At a 1,500-token budget, it improves accuracy by 4.1 pp overall and 8.6 pp on truncated questions, with significant gains in 17 pairs under Holm-corrected bootstrap tests. Offline replay estimates 34.5% of the full-run cost, with a 1.13 pp accuracy loss. A policy selected on training folds saves 71.3% of the estimated full-run cost on held-out folds, with a 0.58 pp accuracy loss. We also find that Jev shows strong question-only accuracy, exceeding 500-token forced answering on average (68.0% vs. 65.5%), although this advantage varies by benchmark. These findings suggest that System-1 decision models can complement LLM reasoning and reduce the need for costly reasoning completion.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.25768 [cs.CL] |
| (or arXiv:2610.25768v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.25768 arXiv-issued DOI via DataCite |
Submission history
From: Zhongyin Wang [view email]
[v1]
Mon, 19 Oct 2026 04:29:19 UTC (270 KB)
[v2]
Sat, 24 Oct 2026 17:52:18 UTC (270 KB)
来源：arXiv cs.CL · arxiv.org