arXiv cs.CL· Jie Jiang, Xing Sun, Ruotian Chen, Jianan Su, Kaixin Shen·· 12 小时前AI 评分31
PPOW：面向投机解码的自适应窗口性能驱动策略优化
Performance-Driven Policy Optimization for Speculative Decoding with Adaptive Windowing
AI 导读
PPOW 是一个强化学习框架，将投机解码的草稿模型优化从 token 级模仿转向窗口级优化，结合成本感知加速奖励、基于分布的接近奖励与自适应散度感知窗口机制。在统一解码协议下，PPOW 在多个模型族和基准上实现 6.29-6.52 的平均接受长度和 3.39-4.36 倍加速。该工作已被 NeurIPS 2026 接收。
正文 · 原文
Abstract:Speculative decoding accelerates LLM inference by having a lightweight draft model propose speculative windows of candidate tokens for parallel verification by a larger target model. In practice, speculative efficiency is often bottlenecked by hard-to-draft positions, where an early mismatch truncates the accepted prefix and invalidates the rest of the speculative window. Most learning-based drafters are still optimized with token-level supervised objectives, even though speculative utility is inherently window-level and prefix-sensitive. We propose PPOW (Performance-Driven Policy Optimization with Adaptive Windowing), a reinforcement learning framework that shifts drafter optimization from token-level imitation to window-level optimization. PPOW combines a Cost-Aware Speedup Reward, a Distribution-Based Proximity Reward, and Adaptive Divergence-Aware Windowing, which prioritizes informative windows with high confidence-weighted draft-target divergence. PPOW achieves average acceptance lengths of 6.29-6.52 and speedups of 3.39-4.36$\times$ across multiple model families and benchmarks under a unified decoding protocol. These results show that performance-driven window-level optimization is a practical approach to improving speculative decoding efficiency.
| Comments: | Accepted at NeurIPS 2026 |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2605.14978 [cs.CL] |
| (or arXiv:2605.14978v3 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2605.14978 arXiv-issued DOI via DataCite |
Submission history
From: Jie Jiang [view email]
[v1]
Thu, 14 May 2026 15:41:57 UTC (345 KB)
[v2]
Fri, 15 May 2026 02:45:40 UTC (354 KB)
[v3]
Tue, 20 Oct 2026 13:08:31 UTC (99 KB)
来源：arXiv cs.CL · arxiv.org