跳到正文
arXiv cs.CL· Yuxuan Liu, Xin Lai, Junyi Li, Pengyuan Lyu, Chenxin Li, Yiduo Guo, Zhengyao Fang, Yang Ding, Yi Zhang, Weinong Wang, Huawen Shen, Xingran Zhou, Liang Wu, Fei Tang, Sunqi Fan, Shangpin Peng, Zheng Ruan, Anran Zhang, Chengquan Zhang, Han Hu, Benyou Wang, Ji-Rong Wen, Rui Yan, Zhengyang Tang·· 10 小时前AI 评分58

PhoneWorld:把真实 App 轨迹转成可验证的手机智能体训练环境

PhoneWorld: From Real-App Trajectories to Dynamic and Verifiable Environments for Phone-Use Agents

AI 导读

PhoneWorld 提出一种基于轨迹的框架,把真实 App 的页面、状态转移和操作记录转成可运行、可重置、可验证的 Android 环境,覆盖 16 个领域的 34 款消费级 App。实验显示,该环境支持端到端在线交互,其轨迹与 AndroidWorld 监督互补,在同等 RL 预算下与真实 App rollout 结合可提升真实手机任务和 AndroidWorld 上的表现。

正文 · 原文

Authors:Yuxuan Liu, Xin Lai, Junyi Li, Pengyuan Lyu, Chenxin Li, Yiduo Guo, Zhengyao Fang, Yang Ding, Yi Zhang, Weinong Wang, Huawen Shen, Xingran Zhou, Liang Wu, Fei Tang, Sunqi Fan, Shangpin Peng, Zheng Ruan, Anran Zhang, Chengquan Zhang, Han Hu, Benyou Wang, Ji-Rong Wen, Rui Yan, Zhengyang Tang

View PDF HTML (experimental)

Abstract:Real applications provide the training setting closest to phone-agent deployment, but are difficult to reset, scale safely, and verify programmatically. Static screenshots and interaction trajectories preserve realistic evidence but cannot generate new experience. We introduce PhoneWorld, a trace-grounded framework that converts such evidence into runnable, resettable, and verifiable Android environments. PhoneWorld induces a usage-weighted interaction skeleton from observed pages, transitions, and state-changing operations; translates it into a behavior-grounded app specification; realizes the specification through an autonomous build--inspect--repair loop; and synthesizes executable tasks with programmatic verifiers. The resulting suite spans 34 consumer-facing apps across 16 domains and supports an audited online benchmark, verified trajectory generation, and online RL through common reset and verification interfaces. Evaluations with diverse general and open-source GUI agents show that PhoneWorld supports reliable end-to-end online interaction and exposes capabilities complementary to AndroidWorld. Controlled SFT experiments further show that PhoneWorld trajectories complement AndroidWorld supervision, transfer across online and offline benchmarks, and become more effective as data volume and app coverage increase. Under a matched RL budget, combining PhoneWorld mock-app rollouts with real-app rollouts improves performance over real-app RL alone on both real-phone tasks and AndroidWorld. Together, these results demonstrate that trace-grounded executable abstraction can bridge realistic mobile behavior and scalable agent learning, turning limited real-app evidence into a growing supply of controllable and verifiable environments for training and evaluation.
Comments: work in progress
Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
Cite as: arXiv:2605.29486 [cs.CL]
  (or arXiv:2605.29486v4 [cs.CL] for this version)
  https://doi.org/10.48550/arXiv.2605.29486

arXiv-issued DOI via DataCite

Submission history

From: Yuxuan Liu [view email]
[v1] Thu, 28 May 2026 07:14:15 UTC (3,147 KB)
[v2] Fri, 14 Aug 2026 05:07:01 UTC (3,170 KB)
[v3] Mon, 28 Sep 2026 15:27:12 UTC (4,663 KB)
[v4] Thu, 19 Nov 2026 18:16:20 UTC (4,664 KB)

来源:arXiv cs.CL · arxiv.org