arXiv cs.CL· Aaryam Sharma·· 11 小时前AI 评分31
TVP：面向投机解码的 TV 距离损失族，接受长度较 CE 提升 9.8%
TVP: TV is Almost All You Need for Speculative Decoding
AI 导读
研究者提出 TVP，一种基于期望接受长度的损失族，其幂参数可在覆盖导向与接受导向目标间插值，将幂从 0 退火到 1 后，在三种 drafter 架构与三种目标模型上取得最高或并列最高的接受长度。
正文 · 原文
Abstract:Speculative decoding accelerates LLM inference with a drafter whose proposed tokens are verified by the target model in parallel. Under lossless speculative sampling, a draft token is accepted with probability one minus the total variation (TV) distance between the target and draft distributions, yet drafters are often trained with cross-entropy (CE) or KL. We first show that under matched and well-tuned training, TV matches or surpasses several published draft-training objectives when fine-tuning, and a fixed TV and KL mixture does the same when training from scratch. We identify two remaining problems, TV's vanishing gradients on undercovered tokens and the prefix products that starve later positions in acceptance-length objectives. To address both, we derive TVP, a loss family based on expected acceptance length, whose power parameter interpolates between coverage and acceptance oriented objectives. Annealing the power from zero to one gives the highest or tied-highest acceptance length across three drafter architectures and three target models, including longer drafts and block verification. TVP improves acceptance length over CE by 9.8% when fine-tuning, and over the public DFlash2 training recipes by 6.2 to 12.9%.
| Comments: | 25 pages, 2 figures, 14 tables |
| Subjects: | Machine Learning (cs.LG); Computation and Language (cs.CL) |
| Cite as: | arXiv:2611.00588 [cs.LG] |
| (or arXiv:2611.00588v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2611.00588 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Aaryam Sharma [view email]
[v1]
Sat, 31 Oct 2026 03:31:48 UTC (139 KB)
来源：arXiv cs.CL · arxiv.org