arXiv cs.AI· Aditya K Surikuchi, Raquel Fern\'andez, Sandro Pezzelle·· 3 天前AI 评分25
多模态基准何在？基础模型识别视频精彩片段的能力研究
Where is the multimodal goal post? On the Ability of Foundation Models to Recognize Highlights in Videos
AI 导读
研究构建了一个基于公开足球精彩集锦的新数据集，用于评估多模态模型区分比赛中重要与非重要子事件的能力。对比多个 SOTA 多模态模型后发现，它们无法可靠捕捉上下文重要性，倾向于依赖单一主导模态，且难以综合多源信息。研究强调需要能动态适应多模态异质性的模块化架构及促进跨模态协同的训练方法。
正文 · 原文
Abstract:Foundation models are used for many real-world applications involving language understanding and generation from temporally-ordered multimodal events. In this work, we study the ability of models to identify the most important sub-events in a video, which is a fundamental prerequisite for summarizing, reasoning over, or narrating multimodal events. Specifically, we focus on football games and evaluate models on their ability to distinguish between important and non-important sub-events in a game. To this end, we construct a new dataset by leveraging publicly available football highlight reels containing the most important moments of a game according to professional experts, without any additional annotation costs. Using our dataset, we compare several state-of-the-art multimodal models and show that they fail to reliably capture contextual importance. Analyses of models beyond standard evaluation metrics reveal their tendency to rely on a single dominant modality and their ineffectiveness in synthesizing necessary information from multiple sources. Our findings underline the importance of modular architectures that dynamically adapt to multimodal heterogeneity and the need for complementary training procedures that maximize cross-modal synergy.
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Computation and Language (cs.CL) |
| Cite as: | arXiv:2601.16333 [cs.CV] |
| (or arXiv:2601.16333v3 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2601.16333 arXiv-issued DOI via DataCite |
Submission history
From: Aditya K Surikuchi [view email]
[v1]
Thu, 22 Jan 2026 21:40:08 UTC (2,294 KB)
[v2]
Thu, 5 Mar 2026 09:51:43 UTC (2,283 KB)
[v3]
Tue, 13 Oct 2026 13:11:47 UTC (2,559 KB)
来源：arXiv cs.AI · arxiv.org