arXiv cs.AI· Yuming Fu, Weijia Wu, Jing Chen, Jiahao Tang, Feifei Chen, Hongyu Zhu, Xin Jin, Alex Jinpeng Wang·· 3 天前AI 评分26
LabInstruct：面向实验室流程的实景教学视频生成基准
LabInstruct: Benchmarking Situated Instructional Video Generation for Lab Procedures
AI 导读
研究团队推出 LabInstruct，一个面向真实实验室的实景教学视频生成基准，涵盖 5 个学科、204 项任务，配有真实参考执行及物体、动作、接触和状态转换的结构化标注。对 8 个前沿图生视频模型的评测显示，视觉上逼真的生成结果往往在流程上并不正确，暴露出视觉真实感与实验教学可靠性之间的显著差距。
正文 · 原文
Abstract:Self-driving laboratories (SDLs) aim to automate the full experimental loop, from scientific decision-making to physical execution. Ideally, AI-generated plans could be carried out directly by robotic systems, but reliable automation remains difficult in complex, open-world laboratory environments, where experiments often involve fine-grained manipulation, long-horizon procedures, and substantial variation across tasks and setups. Humans therefore remain an important execution interface between AI-generated plans and physical experiments, creating a need for clear and effective human-facing experimental guidance. Because laboratory procedures are inherently visual, spatial, and dynamic, video is particularly well suited to communicating apparatus configurations, manipulation actions, temporal dependencies, and state changes. Recent advances in video generation now make it possible to synthesize experimental demonstrations directly from an initial workspace image and a natural-language instruction. However, whether such models can reliably communicate real laboratory procedures has not been systematically studied. We introduce LabInstruct, a benchmark for situated instructional video generation in real laboratories. LabInstruct contains 204 tasks across 5 scientific disciplines, with real reference executions and structured annotations of objects, actions, contacts, and state transitions. Evaluating 8 frontier image-to-video models, we find that visually plausible generations frequently remain procedurally incorrect, revealing a substantial gap between visual realism and the reliability required for experimental instruction.
| Subjects: | Robotics (cs.RO); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.14020 [cs.RO] |
| (or arXiv:2610.14020v1 [cs.RO] for this version) | |
| https://doi.org/10.48550/arXiv.2610.14020 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Yuming Fu [view email]
[v1]
Tue, 29 Sep 2026 12:00:06 UTC (9,741 KB)
来源：arXiv cs.AI · arxiv.org