arXiv cs.CL· Shaden Shaar, Bradon Thymes, Majd Aldaye, Claire Cardie, Bharath Hariharan·· 11 小时前AI 评分38
LF-VideoQA：用长篇心智理论问题评测长视频理解
LF-VideoQA: Evaluating Long-Video Understanding with Long-Form Theory-of-Mind Questions
AI 导读
研究者发布 LF-VideoQA，一个面向长叙事视频的长篇开放式问答数据集，包含 94 部近期电影的 6300 多个问题，每题需跨平均 23 分钟的场景推理，并配有按召回率与精确率打分的题目专属评分标准。
正文 · 原文
Abstract:The hardest part of understanding a long video is not pinpointing any one moment but synthesizing many, e.g., answering a question that requires tracking what develops across hours of content and articulating how the pieces fit together. To date, most long-video QA datasets require only a short answer or a multiple-choice selection, oversimplifying the long-video understanding task to retrieval. We introduce LF-VideoQA, a dataset for long-form, open-ended question answering over long narrative videos built on theory-of-mind questions: a character's belief depends on everything they have seen or been told, so each question demands assembling and reasoning from evidence distributed across the video. The dataset contains more than 6,300 questions on 94 recent films; each requires reasoning over scenes spanning an average of 23 minutes and comes with a question-specific rubric for recall and precision scoring. The dataset is created entirely automatically by annotating screenplays with scene events and each character's first-order beliefs and meta-beliefs, then generating (question, answer, rubric) triples through LLM-based question generation and verification; human studies validate their quality. We evaluate four families of state-of-the-art open-source VLMs across three settings and find the task highly challenging: (i) even an oracle with access to the screenplay falls well short of complete answers; (ii) the best non-oracle model attains barely half of the recall level of the oracle; (iii) unlike on prior long-video benchmarks, captions and subtitles underperform the video itself, indicating that the questions need visual evidence; and (iv) recall is the predominant bottleneck, with answers omitting many required rubric claims.
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Computation and Language (cs.CL) |
| Cite as: | arXiv:2611.00383 [cs.CV] |
| (or arXiv:2611.00383v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2611.00383 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Shaden Shaar [view email]
[v1]
Fri, 30 Oct 2026 21:05:36 UTC (2,838 KB)
来源：arXiv cs.CL · arxiv.org