arXiv cs.AI· Zhaoyu Chen, Hongnan Lin, Yongwei Nie, Fei Ma, Xuemiao Xu, Fei Yu, Chengjiang Long·· 14 小时前AI 评分22
Invert4TVG：用反转任务保持动作理解能力的时序视频定位框架
Invert4TVG: A Temporal Video Grounding Framework with Inversion Tasks Preserving Action Understanding Ability
AI 导读
Invert4TVG 提出一种时序视频定位（TVG）框架，将三类反转任务作为辅助目标以保持模型的动作理解能力：查询中掩码动词补全、动作识别和视频描述。这些任务由原始 TVG 标注派生，并在强化学习框架中与主任务概率整合。实验显示该方法优于现有最优方法，3B 模型在 Charades-STA 上 R1@0.7 提升 7.1%。
正文 · 原文
Abstract:Temporal Video Grounding (TVG) aims to localize video segments corresponding to a given textual query, which often describes human actions. However, we observe that current methods, usually optimizing for high temporal Intersection-over-Union (IoU), frequently struggle to accurately recognize or understand the underlying actions in both the video and query, thus reducing the effectiveness of these methods. To address this, we propose a novel TVG framework that integrates inversion-based TVG as auxiliary objectives to maintain the model's action understanding ability. We introduce three kinds of inversion TVG tasks derived from the original TVG annotations: (1) Verb Completion, predicting masked verbs (actions) in queries given video segments; (2) Action Recognition, identifying query-described actions; and (3) Video Description, generating descriptions containing query-relevant actions given video segments. These inversion tasks are entirely derived from the original TVG tasks and are probabilistically integrated with them within a reinforcement learning framework. By leveraging carefully designed reward functions, the model preserves its ability to understand actions, thereby improving the accuracy of temporal grounding. Experiments show our method outperforms state-of-the-art approaches, achieving a 7.1\% improvement in R1@0.7 on Charades-STA for a 3B model.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2508.07388 [cs.AI] |
| (or arXiv:2508.07388v3 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2508.07388 arXiv-issued DOI via DataCite |
Submission history
From: Zhaoyu Chen [view email]
[v1]
Sun, 10 Aug 2025 15:38:04 UTC (1,659 KB)
[v2]
Fri, 13 Feb 2026 06:37:58 UTC (7,015 KB)
[v3]
Thu, 29 Oct 2026 07:25:14 UTC (7,014 KB)
来源：arXiv cs.AI · arxiv.org