arXiv cs.AI· Jingxing Wang, Teng Wang, Xingyu Lou, Zihan Guo, Jun Wang, Weiwen Liu, Weinan Zhang, Jianghao Lin·· 10 小时前AI 评分38
训练一个能干的 GUI Agent,环境需要多完整?MobileGym 给出三维度实证
How Complete Must an Environment Be to Train a Capable GUI Agent?
AI 导读
研究在 MobileGym 中分别调整奖励覆盖、数据覆盖与功能覆盖三个维度,训练 Qwen3-VL-4B-Instruct 并在原始完整环境中评测。仅奖励单条参考路径会使成功率下降 10.53 点,删除非必要记录同样降低成功率;而裁剪非必要功能反而把成功率从 51.61% 提升到 56.13%。
正文 · 原文
Abstract:Graphical user interface (GUI) agents are widely trained in executable GUI environments. To lower cost, build environments in batches, and reduce manual effort, each work trades away part of the environment through its construction choices. We describe these choices along three dimensions: reward coverage, the goal-reaching trajectories the verifier rewards; data coverage, the application data the agent is exposed to beyond what the task reads or writes; and feature coverage, the application features beyond what the task uses. Existing works set each dimension on engineering grounds, but they do not measure its effect on training. How complete must an environment be to train a capable agent, and along which dimensions? In MobileGym, we adjust each dimension separately, train Qwen3-VL-4B-Instruct, and evaluate every policy in the original full environment. A verifier that rewards only one reference path lowers success by 10.53 points, and removing unneeded records also lowers success. Pruning unneeded features instead raises success from 51.61% to 56.13%, entirely on tasks whose environments it left unchanged. The gain reverses when each episode is weighted equally and vanishes when only episodes on the pruned tasks are balanced. Without pruning, length weighting there reinforces long detour successes that erode skills on other tasks. Following these findings, we build MobileGym-Lite, which keeps reward and data coverage complete and prunes unneeded features. It outperforms Full in the full environment (55.48% versus 51.61% success) with lower construction cost.
| Comments: | 15pages, 6 figures |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2611.23946 [cs.CV] |
| (or arXiv:2611.23946v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2611.23946 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Jingxing Wang [view email]
[v1]
Wed, 18 Nov 2026 06:32:44 UTC (479 KB)
来源:arXiv cs.AI · arxiv.org