arXiv cs.MA· Yanzhe Chen, Ziyi Yang, Jifeng Zhu, Qiming Huang, Ruihe An, Peiyao Xu, Hesen Yang, Runda Liu, Chang Gong, Zhijun Cao, Zechen Bai, Wenzheng Zeng, Yiqi Lin, Guoqiang Liang, Kevin Yuchen Ma, Kevin Qinghong Lin, Mike Zheng Shou·· 13 小时前AI 评分34
多模态具身智能体：从 computer-use 到 robot-use 的统一能力视角 PAPAV 框架
Multimodal Embodied Agents: A Unified Capability-centric Perspective from Computer-Use to Robot-Use
AI 导读
研究者提出 PAPAV 统一能力框架，将多模态具身智能体（MMEA）拆解为感知状态、预判动作效果、规划可行路径、通过接口或本体执行、验证结果五项功能，并据此对比 MMEA、多模态智能体（MMA）与机器人系统（RS）。在 64 个 benchmark 中，显式评估集中在 Act，仅 5 个评估 Anticipate、3 个评估 Verify，后两者大多被任务成功率掩盖。
正文 · 原文
Authors:Yanzhe Chen, Ziyi Yang, Jifeng Zhu, Qiming Huang, Ruihe An, Peiyao Xu, Hesen Yang, Runda Liu, Chang Gong, Zhijun Cao, Zechen Bai, Wenzheng Zeng, Yiqi Lin, Guoqiang Liang, Kevin Yuchen Ma, Kevin Qinghong Lin, Mike Zheng Shou
Abstract:Multimodal agents (MMAs) sustain interaction through reasoning, memory, tools, and feedback, most visibly as computer-use agents, while robotic systems (RSs) couple sensing and actuation under physical dynamics. We define multimodal embodied agents (MMEAs) as goal-directed systems that couple multimodal task reasoning with physical action and revise decisions from the resulting feedback. This raises a central question: what changes when a multimodal agent moves from computer-use to robot-use? We introduce PAPAV, a unified capability-centric framework of five recurring functions: Perceive the current state, Anticipate action effects, Plan a feasible course, Act through an interface or body, and Verify the outcome. Defined by function rather than architecture, these capabilities let PAPAV identify what all three share and then compare MMEAs with MMAs and with RSs. Five physical constraints, from partial observability to unverifiable outcomes, leave fewer choices fixed in advance and less room to reverse errors. Across 64 benchmarks, explicit evaluation centers on Act; only 5 assess Anticipate and 3 assess Verify, leaving both largely hidden behind task success. These constraints also frame the open challenges we identify. PAPAV therefore provides a common basis for designing reliable physical agents and measuring progress across all five capabilities rather than task success alone.
| Comments: | Project website: this https URL |
| Subjects: | Robotics (cs.RO); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV); Multiagent Systems (cs.MA); Multimedia (cs.MM) |
| Cite as: | arXiv:2610.30108 [cs.RO] |
| (or arXiv:2610.30108v1 [cs.RO] for this version) | |
| https://doi.org/10.48550/arXiv.2610.30108 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Ziyi Yang [view email]
[v1]
Wed, 21 Oct 2026 17:52:48 UTC (3,077 KB)
来源：arXiv cs.MA · arxiv.org