arXiv cs.CL· Selim Aktas, Yusuf Erdem Erdem, Ahmed Zeer, Himmet Toprak Kesgin, Muzaffer Kaan Yuce, Mehmet Fatih Amasyali·· 11 小时前AI 评分34
ChainFlow：面向投机解码的少步隐空间流匹配
ChainFlow: Few-Step Latent Flow Matching for Speculative Decoding
AI 导读
ChainFlow 是一种新的投机解码范式，也是首个以目标模型隐藏状态为条件的流匹配 drafter，只需一到两个流步即可完成草稿生成。在 Qwen3.5 模型上、drafter 规模与训练数据匹配时，其在 vLLM 中分别比 EAGLE-3 和 DFlash 快 39% 和 11%。
正文 · 原文
Abstract:Speculative decoding has become a standard way to accelerate LLM inference: a lightweight drafter proposes tokens that the target verifies in one parallel pass. Existing drafters face trade-offs that limits their speedups: autoregressive drafters generate tokens one at a time, so drafting cost grows with draft length, while diffusion drafters generate tokens in parallel but predict each position independently, ignoring inter-token dependencies. We introduce ChainFlow, a new speculative decoding paradigm and the first flow-matching drafter conditioned on the target's hidden states. ChainFlow treats drafting as a short transport over hidden states, needing one or two flow steps instead of the many denoising steps of generation from noise. A variational autoencoder compresses these states so drafting cost barely grows with target size, lightweight feedback heads restore inter-token dependencies, and tree drafting raises acceptance. On Qwen3.5 models, at matched drafter size and training data, ChainFlow outperforms EAGLE-3 and DFlash in vLLM by 39% and 11%, respectively. These gains further widen as drafters are adapted to new languages sequentially. Our ablations cover drafter design and concurrent serving, and we open-source our code and drafter models.
| Subjects: | Artificial Intelligence (cs.AI); Computation and Language (cs.CL) |
| Cite as: | arXiv:2611.08669 [cs.AI] |
| (or arXiv:2611.08669v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2611.08669 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Ahmed Zeer [view email]
[v1]
Thu, 5 Nov 2026 20:26:25 UTC (495 KB)
来源：arXiv cs.CL · arxiv.org