arXiv cs.CL· Jihao Zhao, Qianyu Jiang, Shuaishuai Zu, Pengfei Li, Guanghuan Fang, Ben Feng, Biao Qin, Zhibin Wang, Baochuan Yang·· 3 天前AI 评分27
TreeSpark：为投机解码学习起草待验证的树
TreeSpark: Learning to Draft the Trees You Verify for Speculative Decoding
AI 导读
TreeSpark 是一个统一训练与推理框架，扩展半自回归 DSpark 架构，通过一次主干前向构建父条件候选树，并在训练中重放其扩展、剪枝与预算打包。在 Qwen3-4B 与 Qwen3-8B 上，九个通用基准和六个写作领域的平均接受长度较 DSpark 提升 23.4% 至 31.6%，单请求吞吐提升 15.1% 至 25.0%。
正文 · 原文
Abstract:Parallel drafters such as DFlash and DSpark reduce proposal latency, but early mismatches limit how much of each block can be accepted. Retaining alternatives introduces parent states and candidate choices that reference-conditioned training does not directly supervise. We introduce TreeSpark, a unified training and inference framework extending the semi-autoregressive DSpark architecture. It constructs parent-conditioned candidate trees from one backbone pass and replays their expansion, pruning, and budgeted packing during training. Survival negative log-likelihood weights retained edges by reference-derived path mass, while a capped coverage hinge promotes target-important alternatives near local selection boundaries. Both objectives reuse reference-chain target distributions without additional target forward passes. We evaluate greedy decoding with Qwen3-4B and Qwen3-8B on nine general benchmarks and six writing domains. Across targets and suites, TreeSpark improves mean acceptance length over DSpark by 23.4% to 31.6% and mean single-request throughput by 15.1% to 25.0%. Batched Qwen3-4B throughput improves by 12.6% to 17.2%. At fixed attention capacity and inference policy, the proposed supervision further improves mean acceptance length by approximately 6.4% and mean system throughput by 4.2% to 5.0% on Qwen3-4B.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.20175 [cs.CL] |
| (or arXiv:2610.20175v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.20175 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Jihao Zhao [view email]
[v1]
Wed, 14 Oct 2026 13:21:09 UTC (2,005 KB)
来源：arXiv cs.CL · arxiv.org