arXiv cs.AI· Xiaoyi Chen, Yifei Gao, Yang Xu, Zhenlong Wang, Jian Yan, Jiming An, Xingxing Song, Liqun Deng, Xiaojun Meng, Jiansheng Wei, Yi Zhang, Jitao Sang·· 14 小时前AI 评分31
GUITestScape：面向开放式评估的探索性 GUI 测试基准
GUITestScape: Towards Open-set Evaluation on Exploratory GUI Testing
AI 导读
GUITestScape 是一个覆盖 61 个真实 Android 应用、508 个预设缺陷的交互式基准，同时涵盖交互缺陷与显示缺陷两类。研究团队还提出开放式评估器 GUIJudge，可将智能体的测试轨迹拆解为可独立诊断的能力，性能显著优于所有基线。实验显示检测仍是现有模型的主要瓶颈，将 GUIJudge 的验证器接入现有智能体后无需重训练即可显著提升检测表现。
正文 · 原文
Authors:Xiaoyi Chen, Yifei Gao, Yang Xu, Zhenlong Wang, Jian Yan, Jiming An, Xingxing Song, Liqun Deng, Xiaojun Meng, Jiansheng Wei, Yi Zhang, Jitao Sang
Abstract:Exploratory GUI testing is a particularly demanding setting for MLLM agents: without predefined test scripts, an agent must autonomously navigate an application and discover defects through its own interaction. However, current evaluation falls short on two fronts. First, existing benchmarks focus almost exclusively on interaction defects, leaving display defects outside the evaluation frame. Second, evaluation protocols are bound to predefined defect annotations, collapsing the testing process into a single end-state judgment that conflates qualitatively distinct failure modes. To address these challenges, we present GUITestScape, an interactive benchmark covering 61 real-world Android applications and 508 preset defects spanning interaction and display types, and introduce GUIJudge, an open-set evaluator that decomposes an agent's testing trajectory into independently diagnosable capabilities. Experimental results demonstrate that GUIJudge achieves reliable process-aware evaluation beyond predefined annotations, substantially outperforming all baselines. Benchmarking on GUITestScape further reveals that detection remains the critical bottleneck for existing models across both defect types, and that integrating GUIJudge's verifiers into existing agents significantly boosts their detection performance without retraining.
| Subjects: | Software Engineering (cs.SE); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2605.29532 [cs.SE] |
| (or arXiv:2605.29532v2 [cs.SE] for this version) | |
| https://doi.org/10.48550/arXiv.2605.29532 arXiv-issued DOI via DataCite |
Submission history
From: Xiaoyi Chen [view email]
[v1]
Thu, 28 May 2026 07:47:27 UTC (13,816 KB)
[v2]
Thu, 29 Oct 2026 03:44:30 UTC (15,884 KB)
来源：arXiv cs.AI · arxiv.org