跳到正文
arXiv cs.AI· Yuanhong Zheng, Ruichuan An, Xiaopeng Lin, Yuxing Liu, Sihan Yang, Huanyu Zhang, Haodong Li, Qintong Zhang, Renrui Zhang, Guopeng Li, Yifan Zhang, Yuheng Li, Wentao Zhang·· 10 小时前AI 评分38

PEARL:支持运行时定义概念的个性化流式视频理解模型

PEARL: Personalized Streaming Video Understanding with Runtime-Defined Concepts

AI 导读

PEARL 是一个即插即用的个性化流式视频理解基线,可在视频流任意时间点注册用户定义的概念并检索其历史出现片段。研究同时提出首个该任务基准 PSVU-Bench(132 个视频、带时间戳问答)和训练集 PSVU-10K。

正文 · 原文

Authors:Yuanhong Zheng, Ruichuan An, Xiaopeng Lin, Yuxing Liu, Sihan Yang, Huanyu Zhang, Haodong Li, Qintong Zhang, Renrui Zhang, Guopeng Li, Yifan Zhang, Yuheng Li, Wentao Zhang

View PDF HTML (experimental)

Abstract:Unlike humans, who learn new concepts from a continuous visual stream, existing personalized vision-language models (VLMs) internalize a predefined concept set during training and operate on static images or offline videos. They cannot register a concept that a user defines in the middle of a stream, nor recall the earlier moments in which it appeared. We formalize this task as Personalized Streaming Video Understanding (PSVU) and introduce PSVU-Bench, the first benchmark for this task, which annotates 132 videos with timestamped questions. Its Concept-Definition, Real-Time and Past-Time QA ask a model to register concepts at arbitrary timestamps, recognize their current state, including user-defined actions, and retrieve their historical evidence. We propose PEARL, a strong plug-and-play baseline that stores user-defined concepts apart from archived video clips and rewrites each query with concept descriptions to retrieve the relevant history. Grounding a named concept in video is a skill shared by all concepts. We therefore construct PSVU-10K, a training set of verified concept questions on public internet videos, and train the retriever and answering VLM of PEARL on it once. Concepts defined at test time still require no training. Applied to three architectures without training, PEARL improves Real-Time QA by 15.05% and Past-Time QA by 12.60% on average over the corresponding offline models, and all three surpass existing online video models on both QA types. Training further improves PEARL on Qwen3-VL-8B by 8.66% on Real-Time QA and 9.39% on Past-Time QA, with gains in every category. Code, data and models are available at this https URL.
Comments: Arxiv Submission v2
Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Information Retrieval (cs.IR)
Cite as: arXiv:2603.20422 [cs.CV]
  (or arXiv:2603.20422v2 [cs.CV] for this version)
  https://doi.org/10.48550/arXiv.2603.20422

arXiv-issued DOI via DataCite

Submission history

From: Yuanhong Zheng [view email]
[v1] Fri, 20 Mar 2026 18:47:54 UTC (8,786 KB)
[v2] Wed, 18 Nov 2026 09:30:33 UTC (16,683 KB)

来源:arXiv cs.AI · arxiv.org