arXiv cs.AI· Tae Eun Choi, Sumin Shim, Junhyeok Kim, Seong Jae Hwang·· 13 小时前AI 评分27
用关键帧锚定注意力与重缩放时间 RoPE 实现语义连贯的生成式补帧
Anchoring and Rescaling Attention for Semantically Coherent Inbetweening
AI 导读
研究提出 Keyframe-anchored Attention Bias 与 Rescaled Temporal RoPE，为生成式补帧（GI）的每个中间帧注入来自关键帧和文本的语义与时间引导，无需额外训练即在短序列和长序列上取得最优的帧一致性、语义保真度与节奏稳定性。
正文 · 原文
Abstract:Generative inbetweening (GI) seeks to synthesize realistic intermediate frames between the first and last keyframes beyond mere interpolation. As sequences become sparser and motions larger, previous GI models struggle with inconsistent frames with unstable pacing and semantic misalignment. Since GI involves fixed endpoints and numerous plausible paths, this task requires additional guidance gained from the keyframes and text to specify the intended path. Thus, we give semantic and temporal guidance from the keyframes and text onto each intermediate frame through Keyframe-anchored Attention Bias. We also better enforce frame consistency with Rescaled Temporal RoPE, which allows self-attention to attend to keyframes more faithfully. TGI-Bench, the first benchmark specifically designed for text-conditioned GI evaluation, enables challenge-targeted evaluation to analyze GI models. Without additional training, our method achieves state-of-the-art frame consistency, semantic fidelity, and pace stability for both short and long sequences across diverse challenges.
| Comments: | Accepted to CVPR 2026; Code is released at this https URL |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2603.17651 [cs.CV] |
| (or arXiv:2603.17651v2 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2603.17651 arXiv-issued DOI via DataCite |
|
| Journal reference: | Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 8976-8985, 2026 |
Submission history
From: Tae Eun Choi [view email]
[v1]
Wed, 18 Mar 2026 12:11:02 UTC (36,876 KB)
[v2]
Tue, 20 Oct 2026 08:16:00 UTC (36,861 KB)
来源：arXiv cs.AI · arxiv.org