arXiv cs.CL· Saw S. Lin (Zhiqi Zhang), Jyh-Shing Roger Jang·· 12 小时前AI 评分26
DominoTree：结合 Domino 的条件树结构草稿加速投机解码
DominoTree: Conditional Tree-Structured Drafting with Domino for Speculative Decoding
AI 导读
DominoTree 将 DDTree 的树状草稿与 Domino 的路径依赖因果修正结合，构建无需训练的最佳优先树，在 Qwen3-4B 和 Qwen3-8B 上分别实现平均 5.33x 和 6.89x 的自回归解码加速，峰值达 8.77x。
正文 · 原文
Abstract:Speculative decoding speeds up large language model inference: a small drafter proposes several tokens, and the target model verifies them in one parallel pass. Drafting a tree lets the target verify many alternative continuations at once. DDTree builds such trees from the block-diffusion drafter DFlash, but scores each position independently of the tokens earlier in its branch; Domino conditions on those tokens but drafts only a single chain. DominoTree combines the two: a training-free best-first tree scored by Domino's path-dependent causal correction, restricted to top-M candidates. In a research harness at temperature zero, it averages 5.33x speedup over autoregressive (AR) decoding on Qwen3-4B and 6.89x on Qwen3-8B, peaking at 8.77x; across temperatures it is 20-26% faster than Domino on Qwen3-4B and 27-36% on Qwen3-8B, and 11-20% faster than the stronger of DDTree and its cost-aware variant CaDDTree. A controlled decomposition attributes +10.1% (Qwen3-4B) and +10.7% (Qwen3-8B) accepted length to the correction, and +4.7% and +4.4% to conditioning on the realized path. Served in SGLang on one RTX 5090, it reaches 4.11x and 4.83x AR at batch size 1, 1.17x and 1.09x the speed of Domino's chain, and up to 1.71x AR on 8K-token HELMET summarization, 14-21% above the chain. The lead holds at 2 concurrent requests but reverses from 8 onward, where the chain sustains more goodput despite DominoTree's 28% higher acceptance. DominoTree accepts more tokens than the chain in every serving setting. On the hybrid Gated-DeltaNet Qwen3.6-27B, correct output requires tree-aware recurrent verification; with it, DominoTree matches the chain at batch size 1 (4.58x versus 4.54x AR) and reaches up to 2.11x AR at long context, 15% above the chain.
| Comments: | 31 pages. Submitted to IEEE Access. v4: adds Qwen3.6-27B results, single-GPU serving re-runs, fused tree-builder kernels and tuned budgets; code and data at this https URL |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2607.08642 [cs.CL] |
| (or arXiv:2607.08642v4 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2607.08642 arXiv-issued DOI via DataCite |
Submission history
From: Saw San Lin [view email]
[v1]
Thu, 9 Jul 2026 16:16:35 UTC (115 KB)
[v2]
Fri, 10 Jul 2026 04:07:45 UTC (115 KB)
[v3]
Tue, 18 Aug 2026 06:10:29 UTC (129 KB)
[v4]
Fri, 16 Oct 2026 14:29:03 UTC (108 KB)
来源：arXiv cs.CL · arxiv.org