arXiv cs.AI· Zhuohan Cai, Zhangxuan Gu, Runze Li, Beitong Zhou, Ming Wen, Hongbo Yu, Shuheng Shen, Changhua Meng·· 4 小时前AI 评分32
PACT:通过保留正确计划与修复动作实现 GUI 智能体的计划-动作一致性训练
PACT: Plan-Action Consistency Training by Preserving Correct Plans and Repairing Actions in GUI Agents
AI 导读
PACT 是一种面向 GUI 智能体的训练框架,通过保护已验证的正确计划、修复执行动作来解决计划与动作不一致问题。在 AndroidWorld 上,PACT 将任务成功率从 60.34% 提升至 68.97%(+8.63 pp),AndroidControl High 上的规则化计划-动作一致性从 83.71% 升至 89.74%(+6.03 pp)。
正文 · 原文
Abstract:GUI agents can formulate a correct immediate plan in their reasoning yet emit an action that fails to execute it, creating an execution gap. This recurring mismatch undermines reliable task completion, as a single execution error can derail a multi-step workflow. In outcome-supervised reinforcement learning, broadcasting a negative response-level advantage across all generated tokens risks penalizing valid planning together with erroneous execution. To address this credit-assignment conflict, we introduce PACT (Plan-Action Consistency Training), a training framework that combines reference-aligned plan protection and plan-aligned action repair without altering the original scalar action reward. Using annotated reference actions to identify execution gaps, PACT protects the verified plan by masking negative policy-gradient contributions across its token span. Keeping the generated reasoning unchanged as context, it then favors the reference action over the emitted erroneous action through a preference surrogate with stop-gradient pair weights. Compared with the outcome-only baseline, PACT raises AndroidWorld task success from 60.34% to 68.97% (+8.63 pp), while improving rule-based plan-action consistency on AndroidControl High from 83.71% to 89.74% (+6.03 pp). Adapting PACT to desktop agents also improves task success from 40.92% to 44.17% (+3.25 pp) on the full 369-task OSWorld benchmark. Together, these results establish PACT as an effective credit-assignment framework for bridging execution gaps and improving task completion across mobile and desktop GUI environments.
| Comments: | 21 pages, 11 figures, including supplementary material |
| Subjects: | Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG) |
| Cite as: | arXiv:2611.22591 [cs.AI] |
| (or arXiv:2611.22591v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2611.22591 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Zhuohan Cai [view email]
[v1]
Tue, 17 Nov 2026 11:14:59 UTC (6,188 KB)
来源:arXiv cs.AI · arxiv.org