arXiv cs.CL· Zhaoxiang Feng, Lanxiang Hu, Haoran Yuan, Yu-Yang Qian, Jihui Yang, Peng Zhao, Yibo Zhu, Tajana Rosing, Hao Zhang·· 3 天前AI 评分38
JetSpec++：前瞻并行草稿与自修正的推测解码方法
JetSpec++: Foresight Parallel Drafting with Self Refinements
AI 导读
UC San Diego 等团队提出推测解码方法 JetSpec++，通过块内自修正、块间前瞻草稿与流水线草稿验证三项设计，将草稿模型的边际耗时几乎降为零。相比自回归解码，其墙钟加速最高达 7.45 倍，比 DSpark 快 1.54 倍，在 GB200 上单请求最高 1,361 tokens/s，七个基准平均接受长度 8.94，且保持目标模型输出分布。代码与模型已开源。
正文 · 原文
Authors:Zhaoxiang Feng (1), Lanxiang Hu (1), Haoran Yuan (2), Yu-Yang Qian (3), Jihui Yang (4), Peng Zhao (3), Yibo Zhu (5), Tajana Rosing (1), Hao Zhang (1) ((1) UC San Diego, (2) UIUC, (3) Nanjing University, (4) Meta, (5) StepFun)
Abstract:Speculative decoding's effectiveness is governed by two quantities: the acceptance rate of drafted tokens and the cost of producing them. We show the two objectives can be co-optimized with algorithm-system co-design, and present JetSpec++, which retains a strong learned draft model while hiding its cost almost entirely. JetSpec++ combines three components: intra-block self refinement, which trains the draft model to iteratively correct its own parallel predictions within a block of tokens; inter-block foresight drafting, which begins the next block before the current one is verified; and pipelined drafting and verification, which overlaps the two stages so drafting's marginal wall-clock cost approaches zero. In comparison with state-of-the-art speculative decoding methods, JetSpec++ attains a wall-clock speedup of up to $7.45\times$ over autoregressive (AR) decoding and $1.54\times$ over DSpark, with up to 1,361 tokens per second for a single request on GB200, and an average acceptance length of 8.94 across seven benchmarks, while preserving the target model's output distribution. Our code and models are available at this https URL.
| Comments: | 31 pages, 4 figures, 16 tables. Zhaoxiang Feng and Lanxiang Hu contributed equally. Code and models: this https URL |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.19792 [cs.CL] |
| (or arXiv:2610.19792v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.19792 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Zhaoxiang Feng [view email]
[v1]
Wed, 14 Oct 2026 08:37:27 UTC (324 KB)
来源：arXiv cs.CL · arxiv.org