arXiv cs.AI· Jiushu Li, Haoran Wang, Yumeng Liu, Zhiyang Dou, Yuan Liu, Yuexin Ma, Cheng Lin·· 12 小时前AI 评分31
World2HOSI：面向长时程人-物-场景交互生成的多专家编排框架
World2HOSI: A Multi-Expert Harness for Long-Horizon Human-Object-Scene Interaction Generation
AI 导读
World2HOSI 是一个多专家编排框架，可从 3D Gaussian Splatting（3DGS）场景和自然语言指令生成连续的长时程人-物-场景交互（HOSI）数据。它围绕持久化场景状态构建闭环：编排器规划动作阶段、通过各专家原生条件接口调用专家，并将动作效果写回场景状态。实验与感知用户研究显示，该框架在评测的 HOSI 任务上取得最佳任务完成度和交互质量，同时生成更多样化的数据。
正文 · 原文
Abstract:Long-horizon human-object-scene interaction (HOSI) is central to embodied AI, animation, and game development. Existing human-object interaction (HOI) and human-scene interaction (HSI) models excel in their respective domains, but none alone covers tasks requiring multiple interaction skills. Training unified HOSI models is hindered by scarce and costly data. We introduce World2HOSI, a multi-expert orchestration harness that generates continuous HOSI data from 3D Gaussian Splatting (3DGS) scenes and natural language instructions. Our key insight is that long-horizon HOSI hinges not on a stronger single model but on orchestration. Given a shared, structured scene state and an orchestration harness grounded in it, existing experts can jointly accomplish tasks beyond the reach of any of them alone. World2HOSI realizes this insight as a closed loop around a persistent scene state built from decoupled background and object assets. A harness plans action stages over this state, invokes experts via their native conditioning interfaces, and writes each action's effects back, so that every invocation is grounded in the evolving world. To reduce physical artifacts, bounded residual optimization and contact-driven physics simulation further ensure the physical plausibility of human motion and object manipulation, respectively, while preserving interaction intent. Experiments and a perceptual user study show that World2HOSI achieves the best task completion and interaction quality on the evaluated HOSI tasks, while generating more diverse data.
| Comments: | Project Page: this https URL |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.27725 [cs.CV] |
| (or arXiv:2610.27725v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2610.27725 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Jiushu Li [view email]
[v1]
Tue, 20 Oct 2026 06:43:51 UTC (26,497 KB)
来源：arXiv cs.AI · arxiv.org