arXiv cs.CL· Ruochen Liu, Hao Yan, Ziluowen Luo, Senzhang Wang, Sihong Xie·· 17 小时前AI 评分31
首个推理 token 如何持续影响推理长度：注意力敏感位置的作用
Beyond the Opening: Tracing the First Token's Persistent Influence on Reasoning Length
AI 导读
研究发现，仅替换推理前缀的第一个 token 并继续生成，即使其余前缀 token 不变，剩余推理长度也会明显改变。通过对比不同首 token 条件下的下一 token 分布，敏感度集中在少数生成位置上；阻断这些位置对首 token 的注意力可缩小不同首 token 条件间的长度差距，而在这些位置替换为其他首 token 条件的下一 token 分布同样会显著改变后续推理长度。
正文 · 原文
Abstract:Understanding how individual token choices affect reasoning length is important for characterizing reasoning models. Building on prior findings that changing the first reasoning token (hereafter, the first token) can substantially alter reasoning length, we focus on this initial choice to investigate how its influence operates during subsequent generation. One possibility is that the first token changes the early text that follows, which in turn affects reasoning length. However, replacing only the first token of an existing reasoning prefix and resuming generation substantially changes the remaining reasoning length, even though all other prefix tokens are unchanged. To trace this persistent influence, we compare next-token distributions while varying only the first token and find that sensitivity is concentrated at a small set of generation positions. Blocking attention from these positions to the first token reduces the length gap between first-token conditions. Using next-token distributions from a different first-token condition at these positions also substantially changes subsequent reasoning length. Together, these findings suggest that attention to the first token at a few sensitive positions helps sustain its length effect, illustrating how a local token choice persistently influences a sequence-level outcome.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.34350 [cs.CL] |
| (or arXiv:2610.34350v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.34350 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Ruochen Liu [view email]
[v1]
Sun, 25 Oct 2026 15:03:34 UTC (468 KB)
来源：arXiv cs.CL · arxiv.org