arXiv cs.AI· Mohammed Shabot, Jonathan Adams, Jan Van den Stock, Joost Vennekens·· 3 天前AI 评分20
用 LLM 在下载前预筛视频：面向低成本视频数据集构建
Towards Cost-Efficient Video Dataset Construction with Large Language Models
AI 导读
研究提出在下载视频前用 LLM 基于元数据预筛候选视频，以降低存储、时间和标注成本。LLM 预筛以四名独立标注者的严格多数投票为基准，并用 Gwet's AC1 验证标注可靠性，同时与两种关键词启发式基线对比。团队通过 LOO 与 AORI 机制评估 LLM 作为虚拟标注者的表现，并报告了相对现有方法的成本对比，实现已在 GitHub 开源。
正文 · 原文
Abstract:Research on video classification models has witnessed significant growth recently, accompanied by a significant increase in the demand for datasets to train such models. Existing video dataset curation approaches share a common bottleneck: candidate videos must be downloaded before any relevance judgment is made, which results in storage, time and annotation costs. In this work, we present a filtering step, using Large Language Models (LLMs) to pre-screen videos based on their metadata, before actually downloading these videos, thereby saving on these costs. We evaluate the LLMs against a strict human majority vote derived from four independent annotators and validate the reliability of the annotations through inter-annotator agreement analysis (Gwet's AC1). We compare the LLM pre-filter against two no-LLM baselines, namely two keyword heuristics. Via a Leave-One-Rater-Out (LOO) mechanism and an Add-One-Rater-In (AORI), we evaluate the performance of the LLMs as virtual annotators. Finally, we report cost comparisons relative to existing approaches. To ensure reproducability, we made our implementation available on github.
| Comments: | 18 pages, 9 tables, 1 figure |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.16857 [cs.CV] |
| (or arXiv:2610.16857v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2610.16857 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Mohammed Shabot [view email]
[v1]
Mon, 12 Oct 2026 14:26:44 UTC (6,286 KB)
来源：arXiv cs.AI · arxiv.org