arXiv cs.CL· Yuma Oda, Ryan Mathieu, Roman Knyazhitskiy, Artur Chakhvadze·· 13 小时前AI 评分38
Weaver：用因子化边际构建投机解码提议树，实现 4.08 倍加速
Trees from Marginals: Autoregressive drafting with factorized priors
AI 导读
Weaver 是一种轻量自回归适配器，从因子化 draft 模型的 top-K 边际分布构建提议树，恢复提议 token 间的条件依赖，且无需全词表投影。作者还针对 Gated Delta Net 层推导出无回滚树验证算法，并在 SGLang 中实现优化 CUDA kernel。该方案相比自回归解码实现 4.08 倍加速，并比高度优化的 DFlash 基线高出 16.3%。
正文 · 原文
Abstract:Speculative decoding greatly increases the interactivity of autoregressive language models by trading off computation for extra tokens generated in a single forward pass. Factorized draft models are especially efficient because they predict future-token marginals in parallel, but their independence assumption causes acceptance rates to degrade sharply as the speculative budget grows. We analyze this limitation and introduce Weaver, a lightweight autoregressive adapter that constructs proposal trees from the top-K marginals of a factorized drafter. Weaver restores conditional dependencies between proposed tokens while avoiding a full-vocabulary projection. To support fast verification for models with Gated Delta Net layers, we derive a rollback-free tree-verification algorithm and implement optimized CUDA kernels in SGLang. By combining these model and systems contributions we achieve a 4.08-fold speedup over autoregressive decoding, and outperform a highly optimized DFlash baseline by 16.3%.
| Subjects: | Machine Learning (cs.LG); Computation and Language (cs.CL) |
| Cite as: | arXiv:2607.06763 [cs.LG] |
| (or arXiv:2607.06763v3 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2607.06763 arXiv-issued DOI via DataCite |
Submission history
From: Yuma Oda [view email]
[v1]
Tue, 7 Jul 2026 19:48:36 UTC (956 KB)
[v2]
Sun, 12 Jul 2026 19:52:09 UTC (911 KB)
[v3]
Wed, 21 Oct 2026 16:07:58 UTC (908 KB)
来源：arXiv cs.CL · arxiv.org