arXiv cs.CL· Jaewon Sok, Jewon Yeom, Seonghyeon Park, Jeongjae Park, Hwiyeong Lee, Taesup Kim·· 13 小时前AI 评分39
研究揭示 LLM 闭卷问答错误多源于首 token 召回失败，首 token 提示可修复
On the Tip of the Tongue: First-Token Cueing as a Behavioral Probe of Recall Failures in LLMs
AI 导读
研究者提出 First-Token Cueing 行为探针，用正确答案的首个 token 预填回复，发现许多闭卷错误实为首 token 召回失败。在 16 个开源模型上，仅这一个 token 就能纠正英文多 token 答案中五分之一到一半的错误，并在多语言基准的全部 7 种语言上提升准确率。模型对正确开头比对同等先验概率的错误开头更有信心，但无正确答案时很少能自行找到。
正文 · 原文
Abstract:Large language models often answer factual questions incorrectly in closed-book settings, and such errors are usually attributed to missing knowledge. Yet models often encode answers they fail to produce. We show that many closed-book errors are first-token recall failures: the model commits to a wrong first token, but once given the right one it completes the answer and often recognizes it as correct. We study this with First-Token Cueing, a behavioral probe that prefills the response with the first token of the gold answer. Model uncertainty peaks at the first answer token, where it already predicts correctness, and among gold prefixes of any length the first token alone accounts for the largest share of recovered errors. Across 16 open models, this one token corrects between a fifth and a half of the errors on multi-token answers in English QA and improves accuracy in all seven languages of a multilingual benchmark. Most of the errors it corrects are also solved by the model's own sampled reasoning, a criterion for encoded knowledge, and models recognize correct starts they do not produce: they continue a correct start more confidently than an incorrect start of the same answer type and prior probability, yet rarely find it without the gold answer. These results locate a recoverable class of factual errors at the first decoding step, where inference-time methods may gain the most.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.23400 [cs.CL] |
| (or arXiv:2610.23400v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.23400 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Jaewon Sok [view email]
[v1]
Fri, 16 Oct 2026 12:55:36 UTC (3,305 KB)
来源：arXiv cs.CL · arxiv.org