arXiv cs.CL· Jiaming Fan, Daming Cao, Canchen Huang, Jiale Fu, Jin Zhang, Junjie Gao, Kai Yang, Xiangzhong Luo, Xu Yang·· 3 天前AI 评分22
TreeGraft：面向树状投机解码的自适应多草稿模型嫁接框架
TreeGraft: Adaptive Multi-Drafter Grafting for Tree-Based Speculative Decoding
AI 导读
TreeGraft 是一个多草稿模型框架，让不同成本的草稿模型共同构建一棵共享草稿树，用更强的草稿模型为弱草稿模型的候选重新打分、重选嫁接位置并恢复被遗漏的有希望路径，同时以非破坏性方式整合强模型扩展。它引入从离线价值系统蒸馏的轻量调度器来决定何时调用强草稿模型。在 10 组模型对和 6 个基准上，TreeGraft 平均超越两种固定单草稿模型端点策略中更优者 15.1%，最高增益 26.6%。
正文 · 原文
Abstract:Speculative decoding accelerates large language model inference through a draft-then-verify paradigm. Building on this, tree-structured methods improve inference by organizing proposals into multiple candidate paths, increasing the accepted length. However, existing tree-structured methods use a single drafter for all drafting steps, creating a dilemma: a smaller drafter is fast but yields lower-quality trees, whereas a larger drafter improves tree quality but suffers from high latency. To address this, we propose TreeGraft, a multi-drafter framework in which drafters of different costs jointly construct a shared draft tree. TreeGraft uses the stronger drafter to rescore candidates by updating scores assigned by the weaker drafter, reselect grafting positions, and recover promising paths left unexplored. It also integrates stronger drafter expansions non-destructively, preserving existing branches that may still be accepted by the target model. Together, these designs improve the quality of the shared draft tree. To control the drafting cost, TreeGraft introduces a lightweight scheduler distilled from an offline value system to decide when to call the stronger drafter. Across 10 model pairs and 6 benchmarks, TreeGraft outperforms the better of the two fixed single-drafter endpoint strategies by 15.1% on average, reaching a maximum gain of 26.6%. Our code is available at this https URL.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2608.26112 [cs.CL] |
| (or arXiv:2608.26112v3 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2608.26112 arXiv-issued DOI via DataCite |
Submission history
From: Jiaming Fan [view email]
[v1]
Thu, 28 May 2026 17:52:11 UTC (620 KB)
[v2]
Fri, 28 Aug 2026 04:00:32 UTC (620 KB)
[v3]
Wed, 14 Oct 2026 12:28:01 UTC (648 KB)
来源：arXiv cs.CL · arxiv.org