arXiv cs.AI· Zhang Ziyue, Juwei Jian, Baiyang Song, Yunshan Zhong, Jun Peng, Yiyi Zhou, Rongrong Ji·· 10 小时前AI 评分32
NullTok:用经验零分布做 token 选择,免训练提升长视频理解
NullTok: empirical-null token selection for training-free long video understanding
AI 导读
NullTok 是一种免训练的长视频理解方法,通过为每个 patch 构建经验零分布来校准查询相关性,在 64 个全帧等效(FFE)的视觉 token 预算下,使冻结的 LLaVA-Video-7B 和 Qwen3-VL-8B 在 Video-MME 上分别达到 65.81% 和 68.85%,均超过同预算下所有对比基线。
正文 · 原文
Abstract:Long video understanding remains a daunting challenge for existing multimodal large language models (MLLMs), mainly due to the large number of visual tokens accumulated in long videos. Currently, frame selection and token selection are two main paradigms for addressing this issue, both aiming to select relevant visual information to form a compact input for MLLMs. A natural approach combines these two paradigms by selecting frames before pruning tokens, but noisy frame-level relevance estimates can cause irreversible evidence loss. In this paper, we study this problem from the perspective of assessing the spatiotemporal distribution of query-specific evidence and propose a novel and training-free approach termed NULLTOK. For every patch across all frames, we construct an empirical-null distribution from 64 group-matched irrelevant control queries, capturing the scores expected in the absence of target-specific evidence. Target affinities are calibrated against these same-position references into right-tail Monte Carlo p-values, which are discrete super-uniform under a conditional exchangeability assumption. Calibrated screening and a high-recall semantic guard produce candidate tokens for deterministic visual deduplication and joint spatiotemporal budget allocation. To validate NullTok, we apply it to two representative MLLMs and conduct extensive experiments on a set of video benchmarks. The experimental results demonstrate its effectiveness under a bounded visual token budget. For example, it achieves 65.81% and 68.85% on Video-MME using frozen LLaVA-Video-7B and Qwen3-VL-8B backbones, respectively, at a cap of 64 full-frame equivalents (FFE), without subtitles or audio. These results outperform all evaluated baselines on Video-MME under the same budget for both backbones. Code is available at this https URL.
| Comments: | 8 pages |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.23881 [cs.CV] |
| (or arXiv:2611.23881v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2611.23881 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Ziyue Zhang [view email]
[v1]
Wed, 18 Nov 2026 05:32:52 UTC (2,619 KB)
来源:arXiv cs.AI · arxiv.org