arXiv cs.AI· Paribesh Regmi, Qingshuang Chen, Chi Zhang, Heba Aly, Yelin Kim, Hongda Mao·· 2 天前AI 评分31
面向视频语言模型的内容感知自适应 Token 剪枝
Content-Aware Adaptive Token Pruning for Video-Language Models
AI 导读
研究者提出一种面向视频语言模型的自适应 Token 剪枝策略，根据每个视频的内容动态决定剪枝比例，通过分析 token 嵌入向量的相关性结构量化冗余，为动态场景保留更多 token、对重复或静态内容更激进地剪枝。该方法完全后置，无需额外训练或微调，在仅保留 10% token 时视频描述基准准确率提升 7%，同时计算量减少 95% TFLOPs。
正文 · 原文
Abstract:Vision-language models excel at image and video understanding but suffer from high inference latency due to the need to process thousands of tokens per image, limiting their deployment on resource-constrained edge devices and in real-time surveillance applications. This challenge is further amplified in video processing, where multiple frames must be analyzed simultaneously. Existing token reduction techniques generally rely on a fixed, uniform pruning ratio applied across all inputs, which is suboptimal because the degree of redundancy can vary significantly between different videos, necessitating content-dependent pruning levels to preserve critical information. To address these limitations, we propose an adaptive token pruning strategy specifically designed for video processing. Crucially, the pruning ratio is determined adaptively based on the content of each video. This is achieved by analyzing the correlation structure of token embeddings to quantify redundancy, enabling dynamic token allocation that retains more tokens for dynamic scenes while aggressively pruning repetitive or static content. Importantly, our method is entirely post-hoc and requires no additional training or fine-tuning, while achieving strong empirical gains; notably, it improves accuracy by +7\% on a video captioning benchmark at 10\% token retention, while reducing computation TFLOPs by 95\%.
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2608.03112 [cs.CV] |
| (or arXiv:2608.03112v2 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2608.03112 arXiv-issued DOI via DataCite |
Submission history
From: Paribesh Regmi [view email]
[v1]
Tue, 4 Aug 2026 04:32:48 UTC (2,436 KB)
[v2]
Sun, 25 Oct 2026 20:48:02 UTC (2,496 KB)
来源：arXiv cs.AI · arxiv.org