arXiv cs.AI· Yogesh Kumar·· 2 天前AI 评分22
视频理解中的查询引导定位：QDETRv、TOViT 与 MATR
Where, How, and When: Query-Guided Grounding for Video Understanding
AI 导读
该论文研究查询引导的视频定位三类任务：对象级、关系级与时刻级。对象级提出 QDETRv（Detection Transformer 的查询引导扩展，用于视频中未见对象的一次性定位）与 TOViT（少样本视频对象检测）；关系级引入视频少样本指代关系；时刻级提出 MATR，用双阶段 Dynamic Time Warping 做视频到视频时刻检索，并可扩展至文本到视频检索。
正文 · 原文
Abstract:Understanding video content requires reasoning at multiple levels of granularity, from identifying objects to interpreting their interactions and localizing when specific actions occur. This thesis studies three query-guided grounding tasks that answer where key objects are (object-level grounding), how they interact (relationship-level grounding), and when specific moments occur (moment-level grounding). For object-level grounding, we present QDETRv, a query-guided extension of the Detection Transformer for one-shot localization of unseen objects in videos, which combines query-context fusion, a memory module for temporal consistency, and unsupervised pretraining on synthetic object trajectories. We further propose the Temporal Object-Aware Vision Transformer (TOViT) for few-shot video object detection, which builds on a language-aligned open-set vision encoder and propagates high-confidence object features across frames. For relationship-level grounding, we introduce few-shot referring relationships in videos: given a (subject, predicate, object) query and a few support videos sharing the predicate, the entities are localized by optimizing a T-partite random field using a meta-learned relation network with query-conditioned translational embeddings, Global Semantic Aggregation, Local Localization Aggregation, and belief-propagation-based message passing. For moment-level grounding, we propose the Moment Alignment Transformer (MATR) for video-to-video moment retrieval, which uses dual-stage Dynamic Time Warping for pre- and post-fusion alignment, learnable queries for boundary prediction, and self-supervised pretraining with temporal augmentations; MATR also extends to text-to-video moment retrieval with performance comparable to, or better than, the state of the art. These contributions advance query-guided video understanding across spatial, relational, and temporal grounding.
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.21422 [cs.CV] |
| (or arXiv:2610.21422v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2610.21422 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Yogesh Kumar [view email]
[v1]
Thu, 15 Oct 2026 07:32:57 UTC (11,487 KB)
来源：arXiv cs.AI · arxiv.org