arXiv cs.AI· Hengjie Zhu, Dayan Wu, Xinze Liu, Peng Fu, Zheng Lin, Weiping Wang·· 12 小时前AI 评分34
RECAP：面向多模态投机解码的请求内与持久记忆上下文感知预测回收
RECAP: Context-Aware Prediction Recycling with Request-Local and Persistent Memories for Multimodal Speculative Decoding
AI 导读
RECAP 用一、二、三 token 键存储目标模型预测，请求内上下文记忆隐式绑定当前图像，token 级记忆跨请求保留覆盖，无需训练或辅助草稿模型。在 Qwen2.5-VL-7B 和 LLaVA-1.6-7B 的七个单轮基准上，平均加速分别为 1.66× 和 1.56×，超过最强免训练基线 Token Recycling 的 1.51× 和 1.37×。
正文 · 原文
Abstract:Recycling a target model's own predictions offers a training-free source of proposals for speculative decoding. In vision-language models, however, existing recycling indexes predictions by the last token alone, conflating predictions computed under different images and prefixes; we find that such reuse fails most often at image-sensitive decoding states. Longer keys reduce this aliasing but match fewer states. We introduce \method, which stores target predictions under one-, two-, and three-token keys: contextual memories live only for the current request and are thus implicitly conditioned on its image, while token-level memory persists across requests to retain coverage. At each lookup, entropy-confidence weighting fuses the retrieved predictions according to their key specificity and concentration. Best-first search composes the fused transitions into a candidate tree, supplemented by suffix-matched continuations, and one target pass verifies the tree under greedy decoding. Without training or an auxiliary draft model, \method\ achieves mean speedups over autoregressive decoding of $1.66\speed$ and $1.56\speed$ across seven single-turn benchmarks on Qwen2.5-VL-7B and LLaVA-1.6-7B, exceeding the strongest training-free baseline, Token Recycling ($1.51\speed$ and $1.37\speed$). Our implementation is available in this https URL.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.05894 [cs.AI] |
| (or arXiv:2611.05894v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2611.05894 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Hengjie Zhu [view email]
[v1]
Wed, 4 Nov 2026 03:32:18 UTC (858 KB)
来源：arXiv cs.AI · arxiv.org