arXiv cs.AI· Shijing Hu, Xuancheng Ren, Zhihui Lu, Pan Zhou·· 9 天前AI 评分22
树状推测解码为何丢失目标概率：Exit-Guided 推测解码方法
Where Draft Trees Lose Target Mass: Exit-Guided Speculative Decoding
AI 导读
研究揭示基于草稿分数构建的有限树存在草稿-目标不匹配问题，提出 Tree Exit Verification（TEV）精确验证方法与 ExitTrain 训练方法。TEV 通过一次退出节点决策和一次 bonus token 决策实现层级并行验证，在对话、代码和数学推理任务上，ExitTrain 将平均输出块长度提升 13%，TEV 将验证阶段延迟降低 15%，端到端较 DDTree 加速 14%。
正文 · 原文
Abstract:Tree-based speculative decoding verifies multiple draft continuations in one target-model pass, but finite trees built from draft scores face a fundamental draft-target mismatch. We ask whether better exact verification can increase acceptance on a fixed tree and how target feedback can improve the tree itself. Through a target-flow view, we identify a canonical exit law and prove that one plus target coverage sharply bounds the expected output-block length, including the bonus token, of any exact path verifier. All optimal verifiers share the same exit and bonus-token law, already attained by representative predraw-and-follow and sequential residual verifiers. This yields Tree Exit Verification (TEV), an exact, level-parallel procedure using one exit-node decision and one bonus-token decision. The exit law also identifies missing target probability, providing node-level feedback for Exit-Guided Draft-Tree Training (ExitTrain) on inference-time draft trees. Experiments across dialogue, code, and mathematical reasoning validate fixed-tree equivalence: ExitTrain increases average output-block length by 13%, while TEV reduces verifier-stage latency by 15%, yielding a 14% end-to-end speedup over DDTree. Our results distinguish two opportunities: better draft trees for higher acceptance and more direct verification for lower latency. Code: this https URL.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.11750 [cs.AI] |
| (or arXiv:2610.11750v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.11750 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Shijing Hu [view email]
[v1]
Thu, 8 Oct 2026 11:40:13 UTC (297 KB)
来源：arXiv cs.AI · arxiv.org