arXiv cs.CL· Rotem Elimelech, Michael Shalyt, Ido Kaminer·· 10 小时前AI 评分34
Catch It Where It Happens：在 LLM 推理过程中实时纠错
Catch It Where It Happens: Real Time Error Correction During LLM Reasoning
AI 导读
研究者提出一种在线确定性验证方法，用代码在 LLM 写出推理步骤时实时校验，无需模型调用或占用 GPU，检查失败即指出错误并让模型从原处继续。在符号数学的 ASyMoB 基准上测试三个开源权重模型，准确率提升 1.4-5.6 个百分点，错误答案净减少 1.8-14.9%，token 用量仅增加 6-7%。验证器独立于模型，可与 LLM 进步互补。
正文 · 原文
Abstract:Large language models (LLMs) are increasingly trusted with mathematical, scientific and engineering work, yet their mistakes are often confidently written and hard to spot. A single wrong step tends to propagate, making the model waste inference tokens following a false premise. Existing safeguards either check only the final answer and restart from scratch, or rely on a verifier LLM that competes for GPU capacity and slows down response time. We propose online, deterministic verification: many kinds of reasoning steps can be validated by code, so a step can be verified in real time, as it is written, without any model call or GPU use. Whenever a check fails, we point out the mistake and let the model continue from the same spot. We demonstrate this approach on symbolic mathematics, where derivations already contain explicit expressions that a computer algebra system can check in milliseconds. Tested on the ASyMoB benchmark with three open-weights models, our online validation improves accuracy by 1.4-5.6 percentage points, corresponding to a 1.8-14.9% net reduction in incorrect answers, while increasing token usage by only 6-7%. Because the verifier is independent of the model, the approach complements progress in LLMs rather than competing with it: every new model can benefit immediately, and the benefit grows as deterministic checkers extend to more kinds of reasoning steps.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2611.13107 [cs.CL] |
| (or arXiv:2611.13107v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2611.13107 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Ido Kaminer [view email]
[v1]
Mon, 9 Nov 2026 19:00:40 UTC (461 KB)
来源：arXiv cs.CL · arxiv.org