arXiv cs.AI· Junyi Wu, Zekun Li, Chenhongyi Yang, Chen Guo, Lingchen Yang, Yongan Zhang, Xinyi Zhang, Hongbing Hu, Bartek Rymkowski, Srinath Sridhar, Yan Yan, Xijie Huang·· 3 天前AI 评分37
World-T3：用测试时训练作为实时视频世界模型的持久记忆
World-T3: Test-Time Training as Persistent Memory for Real-Time Video World Models
AI 导读
World-T3 是一个相机控制的视频世界模型，通过测试时训练把过往观测固化进模型参数作为持久记忆，并提出递归最小二乘规则以在线姿态键控方式增量更新记忆。在长时程重访基准上，其重访 PSNR 从 13.52 提升至 18.31 dB，绝对旋转误差从 3.344 降至 1.075，并在单张 NVIDIA B200 GPU 上以 1280 x 704 分辨率维持 20.6 FPS。
正文 · 原文
Authors:Junyi Wu, Zekun Li, Chenhongyi Yang, Chen Guo, Lingchen Yang, Yongan Zhang, Xinyi Zhang, Hongbing Hu, Bartek Rymkowski, Srinath Sridhar, Yan Yan, Xijie Huang
Abstract:Recent advances in video world models have enabled plausible interactive environments, yet long-horizon generation places a fundamental demand on visual memory: the previously observed world must remain consistent as the model continues to generate. Existing autoregressive approaches often implicitly rely on the generative model itself to retain such information, leading to spatial memory decay beyond the active context. We introduce World-T3, a camera-controlled video world model with a test-time training memory that consolidates past observations into dedicated model parameters. To robustly update this memory, we propose a recursive least-squares rule that incrementally absorbs novel visual features via online pose-keyed memory updates. Our TTT memory reads out previously observed scene features from the fast-weight state by camera pose, enabling consistent generation with constant storage and per-step computation regardless of rollout length. To facilitate real-time interactive generation with this robust memory, we further distill the autoregressive model into a few-step generator via dual-teacher distribution matching. On a long-horizon revisit benchmark, World-T3 raises revisit PSNR from 13.52 to 18.31 dB and reduces absolute rotation error from 3.344 to 1.075 relative to the strongest baseline on both metrics, while sustaining 20.6 FPS at 1280 x 704 on a single NVIDIA B200 GPU.
| Comments: | Project Page: this https URL |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Multimedia (cs.MM) |
| Cite as: | arXiv:2610.17434 [cs.CV] |
| (or arXiv:2610.17434v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2610.17434 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Xijie Huang [view email]
[v1]
Mon, 12 Oct 2026 21:52:46 UTC (25,703 KB)
来源：arXiv cs.AI · arxiv.org