arXiv cs.AI· Rituraj Sharma, Weiyuan Chen, Noah Provenzano, Tu Vu·· 14 小时前AI 评分40
PRISM：用过程奖励模型引导推理，突破深度思考的边界
PRISM: Pushing the Frontier of Deep Think via Process Reward Model-Guided Inference
AI 导读
PRISM 是一种由过程奖励模型（PRM）引导的推理算法，通过步骤级验证指导重采样、候选接受与解答聚合。搭配 PRM-score-vote 聚合方法，PRISM 在 gpt-oss-20b 上于 AIME25、HMMT25 和 GPQA Diamond 分别达到 90.0%、75.4% 和 71.4%，超过 gpt-oss-120b。
正文 · 原文
Abstract:DEEPTHINK methods improve reasoning by generating, refining, and aggregating populations of candidate solutions, which enables strong performance on complex mathematical and scientific tasks. However, existing frameworks can still suffer from unreliable correctness signals during inference, which creates a population- enhancement bottleneck where deeper deliberation amplifies errors, suppresses correct minority solutions, and yields weak returns to additional compute. In this paper, we introduce a functional decomposition of DEEPTHINK systems and propose PRISM, a Process Reward Model (PRM)-guided inference algorithm that uses step-level verification to guide resampling, proposal acceptance, and solution aggregation. During refinement, PRISM uses PRM scores to guide population resampling and stochastic local refinement, which concentrates computation on higher-quality reasoning while preserving diversity. When paired with PRM- score-vote, a PRM-guided aggregation method, PRISM is competitive with or outperforms existing DEEPTHINK methods, reaching 90.0%, 75.4%, and 71.4% with gpt-oss-20b on AIME25, HMMT25, and GPQA Diamond, respectively, while exceeding gpt-oss-120b. Additionally, our analysis shows that PRISM produces consistent net-directional correction during refinement and maintains competitive accuracy when the initial population contains few correct candidates.
| Comments: | 28 pages, 22 figures, 4 tables |
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2603.02479 [cs.AI] |
| (or arXiv:2603.02479v2 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2603.02479 arXiv-issued DOI via DataCite |
Submission history
From: Rituraj Sharma [view email]
[v1]
Tue, 3 Mar 2026 00:03:42 UTC (1,212 KB)
[v2]
Thu, 29 Oct 2026 15:18:43 UTC (1,202 KB)
来源：arXiv cs.AI · arxiv.org