跳到正文
arXiv cs.AI· Junxiang Wen, Yinxi Zhao, Jiawei Chen, Jingying Hu, Lijun Wang, Zhen Qin, Shuiguang Deng·· 10 小时前AI 评分31

APS:面向长视频理解的自适应分区帧选择框架

APS: Adaptive Partitioning-Based Frame Selection for Long Video Understanding

AI 导读

APS 是一个免训练的长视频理解帧选择框架,将时间分区建模为复杂度正则化拟合问题,用动态规划联合优化分区边界与局部模型,使分区随 query-frame 相关性自适应调整。

正文 · 原文

View PDF HTML (experimental)

Abstract:Recent advances in large vision-language models (LVLMs) have significantly improved video understanding capabilities, yet processing long videos remains challenging due to the substantial computational cost of extensive visual inputs. Frame selection offers an efficient alternative, but many existing region-based methods rely on fixed or predefined temporal partitions, which may result in suboptimal temporal regions, compromising subsequent budget allocation and frame selection. In this work, we propose Adaptive Partitioning-Based Frame Selection (APS), a training-free framework that adapts temporal partitions to local variations in query-frame relevance. Specifically, APS formulates temporal partitioning as a complexity-regularized fitting problem, jointly optimizing partition boundaries and local models through dynamic programming. The resulting adaptive temporal partitions guide frame budget allocation based on query relevance and temporal extent, enabling informative frame selection under limited visual budgets. Extensive experiments on Video-MME, MLVU, and LongVideoBench demonstrate that APS consistently improves performance across diverse LVLM backbones, achieving the highest average gains over uniform sampling among competing methods. Further analyses validate the effectiveness of adaptive partitioning and demonstrate its generalizability across model scales, similarity encoders, and advanced LVLMs.
Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
Cite as: arXiv:2611.24079 [cs.CV]
  (or arXiv:2611.24079v1 [cs.CV] for this version)
  https://doi.org/10.48550/arXiv.2611.24079

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Junxiang Wen [view email]
[v1] Wed, 18 Nov 2026 09:07:28 UTC (15,321 KB)

来源:arXiv cs.AI · arxiv.org