arXiv cs.AI· Bo-Yu Chen, Zhi Zhou, Yu-Feng Li·· 2 天前AI 评分35
SkillShift-Bench：面向 Web 智能体环境演化下的技能学习基准
SkillShift-Bench: Benchmarking Skill Learning under Environment Evolution for Web Agents
AI 导读
研究者提出 SkillShift-Bench，首个在受控开放环境变化下评估 Web 智能体技能学习的基准，模拟上下文偏移、感知偏移与执行偏移三类场景。实验显示智能体在受控偏移下出现系统性鲁棒性退化：成功率下降、交互成本上升、技能失败更频繁，其中执行偏移压力最大。团队将该问题形式化为非平稳局部可执行性问题，给出动态遗憾分析及一种无需重学技能库的测试时自适应方法。
正文 · 原文
Abstract:Skill learning enables web agents to reuse prior experience for long-horizon interactive tasks. However, learned skills are typically evaluated in closed environments, overlooking the fact that real-world web deployment involves continuous shifts in invocation context, perceptual grounding, and execution feasibility. To this end, we introduce SkillShift-Bench, the first benchmark for evaluating skill learning with controlled open-environment shifts. SkillShift-Bench simulates Contextual Shift, Perceptual Shift, and Execution Shift across three self-hosted web systems, and evaluates agents with diagnostics covering task success, interaction cost, skill use, and skill failure. Comprehensive experiments reveal systematic robustness degradation under controlled shifts, characterized by lower success rates, higher interaction costs, and more frequent skill failures, with Execution Shift exerting the strongest pressure. We further formulate shifted skill execution as a nonstationary local executability problem and provide both a dynamic-regret analysis and a practical test-time adaptation method, demonstrating the feasibility of improving learned-skill execution without relearning the skill library. Code, tasks, environments, documentation, and the leaderboard are available at this https URL.
| Comments: | 70 pages, 8 figures. Accepted to EMNLP 2026 (Main Conference) |
| Subjects: | Artificial Intelligence (cs.AI); Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.33528 [cs.AI] |
| (or arXiv:2610.33528v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.33528 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Bo-Yu Chen [view email]
[v1]
Sat, 24 Oct 2026 08:09:00 UTC (6,810 KB)
来源：arXiv cs.AI · arxiv.org