arXiv cs.AI· Zheng Zhang, Xiaoyang Tang, Shuaiqi Cheng, Jiakai Huang, Zhongbo Wu, Ziyi Zhang, Yuan Si, Zeyu Liu, Weibo Gao, Shunyu Liu, Shan You, Fei Wang, Xiaogang Wang, Dacheng Tao·· 12 小时前AI 评分38
WorldFailBench：世界模型能理解策略失败吗？
WorldFailBench: Do World Models Understand Policy Failures?
AI 导读
研究者提出 WorldFailBench，用失败原因、失败阶段、失败边界三项任务检验世界模型是否忠实模拟机器人策略的失败过程。基准覆盖 LIBERO、RoboTwin 仿真环境及 G1 人形机器人真实操作任务，对七个世界模型的实验显示它们在理解策略失败上存在明显差距。
正文
Authors:Zheng Zhang, Xiaoyang Tang, Shuaiqi Cheng, Jiakai Huang, Zhongbo Wu, Ziyi Zhang, Yuan Si, Zeyu Liu, Weibo Gao, Shunyu Liu, Shan You, Fei Wang, Xiaogang Wang, Dacheng Tao
Abstract:Robot policy evaluation informs deployment decisions and policy improvement, but physical testing and simulator preparation require substantial effort. World models offer a complementary route through closed-loop interaction in generated environments. Recent benchmarks have demonstrated their potential as policy evaluators through outcome-based evaluation (i.e., assessing task success or failure). However, such evaluation does not establish whether world models faithfully simulate how policies fail. Misrepresenting failures can lead to incorrect diagnoses of policy weaknesses and make policies appear ready for deployment even when small scene changes cause them to fail. We therefore ask: do world models faithfully simulate how and at which stages policies fail, and under what conditions they transition from success to failure? To answer this question, we introduce WorldFailBench with three evaluation tasks. Failure reason tests whether models reproduce the observable errors policies make. Failure stage assesses whether failures concentrate at the same stages of full-task execution. Failure boundary examines whether models reproduce the conditions under which policies transition from success to failure under controlled scene changes in simulation. The benchmark covers manipulation tasks in the LIBERO and RoboTwin simulation environments and real-world manipulation tasks performed by the G1 humanoid robot. Experiments across seven world models reveal substantial gaps in their understanding of policy failures, providing guidance for developing more reliable world models for policy evaluation.
| Subjects: | Robotics (cs.RO); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.33565 [cs.RO] |
| (or arXiv:2610.33565v1 [cs.RO] for this version) | |
| https://doi.org/10.48550/arXiv.2610.33565 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Zheng Zhang [view email]
[v1]
Sat, 24 Oct 2026 09:33:52 UTC (6,632 KB)
来源：arXiv cs.AI · arxiv.org