arXiv cs.CL· Yongchao Long, Chunrui Zheng, Yuxuan Zhou, Guangkun Nie, Tian-Chi Ma, Chao Pang, Shuohuan Wang, Yuchen Ding, Xianbin Wen, Dianhai Yu, Junyuan Shang, Yuxi Zhou, Shenda Hong·· 12 小时前AI 评分31
投机解码的接受长度何时值得更高成本？一项受控研究
When Does Better Acceptance Pay? A Controlled Study of Speculative Decoding
AI 导读
一项受控研究重新测量了已发布 checkpoint 并在共享目标模型生成数据上重训六种 drafter 方案，发现统一训练与推理条件后，已发表对比中的接受长度差距大幅缩小。接受长度可通过近乎零成本的目标与上下文调整提升，DCore 作为强基线几乎不增加额外草稿开销。收益是有条件的：任务不确定性与服务条件会影响接受回报、执行成本和最优 drafter 与预算选择。
正文 · 原文
Authors:Yongchao Long, Chunrui Zheng, Yuxuan Zhou, Guangkun Nie, Tian-Chi Ma, Chao Pang, Shuohuan Wang, Yuchen Ding, Xianbin Wen, Dianhai Yu, Junyuan Shang, Yuxi Zhou, Shenda Hong
Abstract:The throughput gains of speculative decoding depend on the trade-off between acceptance length and execution latency. We ask when additional acceptance is worth the cost of obtaining it. We establish a comparison protocol spanning training, inference, and evaluation, remeasuring released checkpoints and retraining six drafter recipes on shared target-generated data across ten benchmarks and two target models. Alignment changes the picture: gaps that appear large in published comparisons shrink substantially under shared training and inference conditions. This narrowing motivates two deeper questions: how is acceptance improved at different costs, and when do these gains improve throughput? We find that acceptance can be improved through nearly free objective and context changes or more expensive token-to-token conditioning. DCore illustrates the former route as a strong baseline with little additional measured drafting overhead. The payoff is conditional: task uncertainty and serving conditions affect acceptance returns, execution costs, and the preferred drafter and budget. These findings inform future drafter design and deployment choices.
| Comments: | 24 pages, 11 figures. Project page: this https URL |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.24993 [cs.CL] |
| (or arXiv:2610.24993v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.24993 arXiv-issued DOI via DataCite |
Submission history
From: Yongchao Long [view email]
[v1]
Sun, 18 Oct 2026 03:52:06 UTC (159 KB)
[v2]
Tue, 20 Oct 2026 07:26:20 UTC (159 KB)
来源：arXiv cs.CL · arxiv.org