跳到正文
arXiv cs.AI· Xiaolin Lin, Kai-Chiang Wu, Bing Li, Grace Li Zhang·· 10 小时前AI 评分38

SkimPrefill:让小模型先读,加速长交错图文输入的预填充

Let a Small Model Read First: Faster Prefill for Long Interleaved Image-Text Inputs

AI 导读

SkimPrefill 让小型混合模型 Qwen3.5-0.8B 先以半分辨率图像读完整个输入,仅用早期层的一个注意力头对图像或段落单元排序,再把排名靠前的单元作为更短的提示词交给目标模型。

正文 · 原文

View PDF HTML (experimental)

Abstract:Vision-language models (VLMs) can answer questions about long interleaved inputs, such as documents of many pages and collections of hundreds of images, but they are slow to start: on inputs of about 100K tokens, the prefill takes most of the wait for the first output token. A common approach prunes the input by scoring its tokens with the target model itself or with a small model of the same family that first writes an answer. However, this scoring takes much of the time that pruning saves. We find that the evidence for a question lies in a few whole units (images or paragraphs), and that a small hybrid model finds them with one attention head of an early layer, even at half image resolution. Based on these findings, we propose SkimPrefill, which lets the small hybrid model read the whole input first, up to that early layer, with images at half resolution. Most layers of the small hybrid model (Qwen3.5-0.8B) run in linear time, so this pass is short. SkimPrefill then ranks every unit with that head and gives the target the top units as an ordinary shorter prompt. One untrained selector serves targets of different model families. Extensive experiments on MMLongBench at 128K tokens, with 19 datasets of five task types, show the effectiveness of SkimPrefill. With 10% of the input tokens kept, SkimPrefill speeds up the prefill of Qwen3.5-9B, Gemma-4-12B and Qwen3-VL-32B by 3.7 times, 11.6 time and 8.2 times while maintaining their average accuracy.
Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
Cite as: arXiv:2611.24431 [cs.CV]
  (or arXiv:2611.24431v1 [cs.CV] for this version)
  https://doi.org/10.48550/arXiv.2611.24431

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Grace Li Zhang [view email]
[v1] Wed, 18 Nov 2026 13:49:02 UTC (1,265 KB)

来源:arXiv cs.AI · arxiv.org