arXiv cs.AI· Muyang Zhao, Chunhao Tian, Qi Qi, Hao Sun·· 3 天前AI 评分29
ROI-Reasoning：通过预计算元认知实现推理的理性优化
ROI-Reasoning: Rational Optimization for Inference via Pre-Computation Meta-Cognition
AI 导读
ROI-Reasoning 是一个两阶段框架，让 LLM 在严格全局 token 预算下学会预判问题难度并策略性分配算力。它先通过元认知微调让模型预测推理成本并判断问题是否值得尝试，再用理性感知强化学习优化长程分配策略。在数学推理、选择题问答和代码生成任务中，该方法在紧张预算下提升得分并降低参考序后悔，数学任务上相对同骨干最强基线最高提升 417%。
正文 · 原文
Abstract:Large language models (LLMs) can achieve strong reasoning performance with sufficient computation, but they do not inherently know how much computation a task requires. We study budgeted inference-time reasoning for multiple problems under a strict global token constraint and formulate it as an Ordered Stochastic Multiple-Choice Knapsack Problem (OS-MCKP). This perspective highlights a meta-cognitive requirement---anticipating problem difficulty and allocating computation strategically. We propose ROI-Reasoning, a two-stage framework that endows LLMs with intrinsic, budget-aware rationality. First, Meta-Cognitive Fine-Tuning teaches models to predict reasoning cost levels and decide whether a problem is worth attempting before generation. Next, Rationality-Aware Reinforcement Learning optimizes sequential decision making under a hard token budget, allowing models to learn long-horizon allocation strategies. Experiments across mathematical reasoning, multiple-choice question answering, and code generation show that ROI-Reasoning improves scores and generally reduces reference-order regret under tight budgets, with up to a \textbf{417\%} relative score improvement over the strongest same-backbone baseline in mathematics.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2601.03822 [cs.AI] |
| (or arXiv:2601.03822v2 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2601.03822 arXiv-issued DOI via DataCite |
Submission history
From: Muyang Zhao [view email]
[v1]
Wed, 7 Jan 2026 11:30:55 UTC (1,187 KB)
[v2]
Tue, 13 Oct 2026 15:17:27 UTC (1,631 KB)
来源：arXiv cs.AI · arxiv.org