arXiv cs.CL· Jiaming Xu, Hanzhen Wang, Tianlang Zhao, Jiewen Xiao, Chengze Yuan, Jiayi Pan, Yongkang Zhou, Jiancai Ye, Yu Wang, Guohao Dai·· 12 小时前AI 评分38
DLite：面向 Agentic AI 的无历史投机解码框架
DLite: History-free Speculative Decoding for Agentic AI
AI 导读
DLite 是一个面向 Agentic AI 的无历史投机解码框架，直接复用目标模型的上下文隐藏状态做局部投机预测，无需显式维护历史上下文。在通用、长上下文和 Agentic 负载上，DLite 实现 2-6 的接受长度，最高比 SOTA 方法 DSpark 高出 70%。现有草稿模型在 Agentic 负载下接受长度最多下降 80%，而维护历史 KV cache 会带来显著系统开销。
正文 · 原文
Abstract:Speculative decoding has become a mainstream technique for accelerating large language model (LLM) inference by using a lightweight draft model to generate candidate tokens that are subsequently verified by the target model. However, agentic workloads introduce long and continuously evolving contexts and frequent multi-turn interactions. Under such workloads, we observe that existing draft models suffer from up to an 80% degradation in acceptance length, while maintaining their historical KV cache introduces substantial system overhead. In this paper, we identify the growing historical context as a common source of both problems and introduce DLite, a history-free speculative decoding framework for agentic AI. DLite directly reuses contextual hidden states from the target model and performs local speculative prediction without explicitly maintaining the historical context, transforming an increasingly complex modeling problem into local simple prediction problem. A lightweight sequential head further restores intra-block causal dependencies with low overhead. To effectively train this heterogeneous history-free architecture, we introduce a tailored two-stage training strategy. Experiments on general-purpose, long-context and agentic workloads show that DLite achieves 2-6 acceptance length and outperforms the state-of-the-art speculative decoding method, DSpark, up to 70%.
| Comments: | 14 pages, 5 figures |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.23787 [cs.CL] |
| (or arXiv:2610.23787v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.23787 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Jiaming Xu [view email]
[v1]
Fri, 16 Oct 2026 17:18:54 UTC (795 KB)
来源：arXiv cs.CL · arxiv.org