ProAgent：持续观察与及时干预的主动式 Physical AI 系统
Proactive Physical AI through Continuous Observation and Timely Intervention
研究者提出"主动式 Physical AI"问题并开发 ProAgent 端到端系统，由判断是否需要及何时干预的 StreamingJev 主动时机模型与生成响应或调用设备能力的 Response Model 组成。团队还构建了将视频转为主动时机监督信号的自动化数据管线，实验显示其干预时机更准、误触发少于基线，真机评测验证了识别新兴需求并转化为物理动作的能力。
Authors:Zhenhua Yang, Jiafeng Liang, Xingyu Zhou, Yi Jiang, Xiayu Cao, Zui Chen, Luqi Wang, Haiyang Sun, Yufei Xiang, Chang Li, Rongchuan Mu, Bowen Chi, Zihe Tong, Mingyu Li, Huijie Ma, Yiqiao Yang, Baoqi Ren, Zhiming Ma, Zhihao Zhu, Jiahao Ji, Zhuoyao Wang, Qianyu Wang, Xiangyu Liu, Wang Zixiang, Shunliang Zhang, Ming Liu, Xiaocheng Feng, Duyu Tang, Bing Qin
Abstract:Physical AI is commonly formulated around an explicitly specified task: given a goal or instruction, a system perceives, reasons, and acts. In many real-world settings, however, important tasks emerge from continuously changing world states before a person explicitly formulates them. We study this complementary problem as Proactive Physical AI, where an AI system continuously observes the physical world, identifies situations that warrant intervention, and initiates appropriate actions at the right time. We develop ProAgent, an end-to-end system built around two components: a Jev-style proactive timing model (dubbed StreamingJev) that determines whether and when intervention is needed, and a Response Model that generates responses for people or invokes available device capabilities for physical execution. To support learning at scale, we develop an automated data pipeline that converts videos into supervision for proactive timing. Experiments demonstrate strong proactive performance under both explicit objectives and standing roles, combining more accurate intervention timing with fewer false triggers than the source-backbone baseline. Real-robot evaluations further demonstrate ProAgent's ability to identify emerging needs and translate them into appropriate physical actions.
| Comments: | 33 pages, 5 figures, including appendices |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Robotics (cs.RO) |
| Cite as: | arXiv:2610.16967 [cs.CV] |
| (or arXiv:2610.16967v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2610.16967 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Zhenhua Yang [view email]
[v1]
Mon, 12 Oct 2026 15:33:34 UTC (5,128 KB)
来源：arXiv cs.AI · arxiv.org