arXiv cs.AI· YaJie Yin·· 14 小时前AI 评分38
无形式化答案场景下的可训练边界保持：证据充分性判断中奖励训练的受控 knock-in 研究
Trainable Boundary-Keeping Where No Formal Answer Exists: A Controlled Knock-In Study of Reward Training in Evidence-Sufficiency Judgment
AI 导读
一项预注册、同基座、受控 knock-in 的 14B 同族模型研究显示，采用 SFT 后接 GRPO 的边界保持方案，可将证据充分性判断中的自发正确拒答率从 25% 提升至 100%（每组 40 例，Wilson 95% CI 下限 91.2%，与基线区间 [14.2%, 40.2%] 不重叠），而打乱配对的安慰剂组降至 5% 自发正确、91.1% 误拒。
正文 · 原文
Abstract:Reasoning-tuned and reward-trained models are often assumed to verify more reliably because they reason longer or receive correctness signals. We test this in evidence-sufficiency judgment: deciding whether available information actually supports a causal claim, a domain with no decidable correctness reward. Using a pre-registered, same-base, controlled knock-in design, we compare a boundary-keeping recipe (SFT then GRPO) with its own baseline, a scrambled-pairing placebo, and a general-purpose reasoning model under an action-pressure protocol with no evaluation invitation. All load-bearing outcomes are rated by two separate model-rating sessions masked to arm labels; automatic judges only trigger. The trained arm raises spontaneous correct refusal from 25% to 100% (40 per arm; Wilson 95% CI lower bound 91.2%, non-overlapping with the baseline interval [14.2%, 40.2%]), while the scrambled placebo collapses (5% spontaneous / 7.5% invited correct; 91.1% false refusal). The effect is boundary-specific rather than blanket refusal: automatic false refusal on 281 well-formed claims is 1.1% (Wilson upper bound 3.09%), and a targeted 18-item arm-(c) recheck (3 flagged plus 15 accepted) yields 0/18 and 1/18 false refusals in two sessions. The dissociation is the main result: final boundary-keeping is trainable without an inference-time verifier, while observed active numeric/calculation checking in the H3 assay remains near zero; SFT saturation makes the GRPO increment not separately identifiable. A pre-registered mechanism adjudication favors reward internalization of final-action status (H2, best-supported, not proven). The scope is a controlled 14B same-family study: we do not claim proof, cross-family generality, or independent human-expert validation; the rating sessions are model sessions, not human reviewers.
| Comments: | 14 pages, 3 figures. Aggregate data/figure code: this https URL (code MIT; data/text/figures CC BY 4.0). Writing/implementation assisted by AI agents; the author designed the study and is responsible for the final text |
| Subjects: | Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.35226 [cs.AI] |
| (or arXiv:2610.35226v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.35226 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: YaJie Yin [view email]
[v1]
Mon, 26 Oct 2026 11:16:28 UTC (98 KB)
来源：arXiv cs.AI · arxiv.org