arXiv cs.AI· Yijing Lin, Zhendong Mao, Mengqi Huang, Yuxin Wang, Yongdong Zhang·· 13 小时前AI 评分34
FrameForge：利用视频基础模型先验实现统一图像生成
FrameForge: Harnessing Priors of Video Foundation Models for Unified Image Generation
AI 导读
FrameForge 通过输入自适应的虚拟时间关系，把视频基础模型用于统一条件图像生成。其 Mixture of Temporal Experts（MoTE）动态选择不同虚拟间隔的时间 RoPE 专家来建模目标与条件的距离，Chain of Temporal Reasoning（CoTR）则用潜在中间状态和推理瓶颈注意力掩码建模条件到目标的变换。
正文 · 原文
Abstract:Unified conditional image generation requires a single model to preserve different aspects of a visual input while producing a target image that follows textual guidance. Pretrained video models provide a promising foundation by encoding visual transformations across time, yet existing approaches typically assign the condition and target to fixed temporal positions. We observe that condition-target relationships vary substantially across generation tasks, while video models naturally encode visual transformations at different temporal scales. This motivates us to model conditional generation through input-adaptive virtual temporal relations. We propose FrameForge, which captures two complementary aspects of such relations. Mixture of Temporal Experts (MoTE) models how far the target lies from the condition by dynamically selecting temporal RoPE experts with different virtual intervals. Chain of Temporal Reasoning (CoTR) models how the condition is transformed into the target through a latent intermediate state, with a reasoning-bottleneck attention mask encouraging information propagation through this pathway. Experiments on DreamBench, COCO2017, and GEdit-Bench show that FrameForge achieves superior performance in subject-driven generation, canny-to-image generation, and image editing among the compared methods.
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.36545 [cs.CV] |
| (or arXiv:2610.36545v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2610.36545 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Yijing Lin [view email]
[v1]
Tue, 27 Oct 2026 02:59:22 UTC (11,075 KB)
来源：arXiv cs.AI · arxiv.org