arXiv cs.CL· Jiwon Jeon, Myungsik Cho, Jeonghye Kim, Youngchul Sung·· 6 天前AI 评分32
UPTURN：学习在何处重定向 LLM 推理以实现自我纠错
Inject a Turn, Not a Token: Learning Where to Redirect LLM Reasoning for Self-Correction
AI 导读
UPTURN 通过在冻结编码器上训练 130K 参数 MLP 头定位干预位置，以新用户轮次注入指令并重新生成后续内容，无需微调 LLM。在四个推理模型和六个数学基准上，该方法在所有设置中均优于 32-rollout 基线，并在匹配纠错预算下超越自我纠错与反思 token 注入基线。它可跨推理模型迁移并泛化至 GPQA-Diamond，且与 GRPO 及自我纠错方法互补。
正文 · 原文
Abstract:Large language models struggle to reliably identify and correct errors in their own reasoning. Unlike prior training-based approaches that fine-tune an LLM to generate revisions or feedback, we learn where to intervene in failed rollouts and determine how to redirect them. To choose where to intervene, we train a 130K-parameter MLP head on a frozen encoder using only final-answer correctness. For redirection, we find that under a fixed generation budget, the tested inline injections fail to reliably redirect reasoning, whereas presenting the same instruction in a new user turn changes the model's course. The resulting correction framework, UPTURN (User-Prompt TURN injection at a learned position), requires no LLM fine-tuning: it retains a selected prefix, injects an instruction as a new user turn, and regenerates the suffix. Across four reasoners and six math benchmarks, UPTURN improves average accuracy in every setting over a 32-rollout baseline. It outperforms self-correction and reflection-token injection baselines under matched correction budgets, transfers across reasoners and to GPQA-Diamond without retraining, and complements GRPO and self-correction methods.
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.13672 [cs.CL] |
| (or arXiv:2610.13672v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.13672 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Jiwon Jeon [view email]
[v1]
Fri, 9 Oct 2026 14:37:51 UTC (1,694 KB)
来源：arXiv cs.CL · arxiv.org