arXiv cs.AI· Chenyuan Zhou, Yining Hong, Huang Huang, Yunfan Jiang, Wenlong Huang, David Held, Vitor Campagnolo Guizilini, Yue Wang, Li Fei-Fei, Jiajun Wu, Leonidas Guibas, Yejin Choi·· 2 天前AI 评分34
T2-Z：面向机器人世界模型的环境隐变量测试时优化
T2-z: Test-Time Optimization of Environment Latents for Robotic World Models
AI 导读
T2-Z 通过为机器人世界模型附加可学习的环境隐变量，在测试时冻结世界模型、仅用一次或多次交互轨迹优化该隐变量，从而适应未见过的环境配置。在 7 个模拟设置和 4 项真实机器人任务中，物体预测误差指标从 15.20 降至 7.00，动作成功率从 33.73% 提升至 72.96%。消融实验显示，T2-Z 仅凭单条测试时交互轨迹即可高效推断环境隐变量。
正文 · 原文
Authors:Chenyuan Zhou, Yining Hong, Huang Huang, Yunfan Jiang, Wenlong Huang, David Held, Vitor Campagnolo Guizilini, Yue Wang, Li Fei-Fei, Jiajun Wu, Leonidas Guibas, Yejin Choi
Abstract:Predicting how the world responds to an action requires more than what can be seen: robots must account for environmental factors that are neither directly observable nor explicitly specified, from physical properties such as friction and mass to human-defined environmental rules. We introduce T2-Z for test-time optimization of environment latents, which augments a robotic world model with a learnable environment latent to condition world predictions. This compact latent captures underlying environmental variation without supervision of what environmental factors exist or what values they take. During training, T2-Z alternates between two stages: a world-model optimization stage that fixes the latents and learns general action-conditioned dynamics across environments; and an environment-latent optimization stage that freezes the world model and optimizes the conditioning latents, allowing latent structure to emerge purely through world prediction. At test time, the world model remains frozen and a newly initialized environment latent is optimized from one or more test-time interactions, adapting the model to previously unseen environmental configurations. Across 7 simulated settings and 4 real-world robotic tasks, T2-Z reduces object prediction error metric from 15.20 to 7.00 and raises action success from 33.73% to 72.96%. Its emergent latents capture structured representations of individual and compounded factors across diverse visual scenes. Ablations further show that T2-Z can efficiently infer environment latents from as little as a single test-time interaction trajectory.
| Subjects: | Robotics (cs.RO); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV) |
| Cite as: | arXiv:2610.22364 [cs.RO] |
| (or arXiv:2610.22364v1 [cs.RO] for this version) | |
| https://doi.org/10.48550/arXiv.2610.22364 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Yining Hong [view email]
[v1]
Thu, 15 Oct 2026 17:59:11 UTC (34,092 KB)
来源：arXiv cs.AI · arxiv.org