跳到正文
arXiv cs.AI· Yuhao Wang, Zelin Zang·· 4 小时前AI 评分38

ABACUS:用可复用程序为冻结的判别模型构建多步推理

ABACUS: Program-Guided Multistep Reasoning over Frozen Judgment Models

AI 导读

ABACUS 框架通过离线生成程序、在线由冻结判别模型回答局部语义问题,把局部判断组合成多步推理答案。在三个 BIG-Bench Hard 任务上,经开发集参考一致性优化后的三个程序在 600 道留出题上达到 99.67% 宏平均准确率,而直接使用 Kev 为 64.00%、仅输出最终答案的 Qwen3.5-4B 为 68.80%。

正文 · 原文

View PDF HTML (experimental)

Abstract:Judgment models answer semantic questions by scoring a finite set of candidate outcomes. Can reusable programs turn these local decisions into powerful multistep reasoning? We introduce \method, a framework that jointly constructs judgment schemas and composition programs for frozen judgment models. A generative teacher designs the procedure offline; the judgment model resolves local semantic questions online, and the saved program composes its decisions into an answer. We study three allocations of design effort between humans and the teacher, focusing on construction through a shared interface. We instantiate the framework with Kev. On three selected BIG-Bench Hard tasks, nine independently constructed programs solve all 450 executions of 150 development inputs. A separate deployment study freezes three programs after development-set reference-agreement refinement and evaluates them on 600 held-out questions. These programs achieve near-perfect 99.67\% macro accuracy, versus 64.00\% for direct Kev and 68.80\% for final-answer-only Qwen3.5-4B. They attain comparable observed accuracy to Qwen's unrestricted response policy (99.07\%) with an \textbf{8.43-fold online speedup}: 0.255 versus 2.150 seconds per question. These results show how reusable programs can substantially expand the reasoning capability of a compact judgment model while keeping its parameters fixed.
Subjects: Artificial Intelligence (cs.AI)
Cite as: arXiv:2610.25963 [cs.AI]
  (or arXiv:2610.25963v2 [cs.AI] for this version)
  https://doi.org/10.48550/arXiv.2610.25963

arXiv-issued DOI via DataCite

Submission history

From: Zelin Zang [view email]
[v1] Mon, 19 Oct 2026 07:57:48 UTC (314 KB)
[v2] Tue, 17 Nov 2026 09:07:38 UTC (494 KB)

来源:arXiv cs.AI · arxiv.org