arXiv cs.CL· Sihan Yang, Weichen Bai, Ruichuan An, Yiming Cheng, Zhiyuan Feng, Tianyi Jiang, Renye Yan, Ning Ding, Yifan Zhang, Renrui Zhang, Yehui Tang, Wentao Zhang·· 13 小时前AI 评分31
EnvSched：面向 rollout 高效智能体强化学习的环境感知调度
EnvSched: Environment-Aware Scheduling for Rollout-Efficient Agent Reinforcement Learning
AI 导读
研究者提出在线调度器 EnvSched，通过从 rollout 历史学习，将环境—任务对组织为双关系图并预测任务内奖励方差，优先调度高方差任务与欠观察环境。在 Qwen3-8B 的强化学习实验中，EnvSched 训练 40 步后在 BFCL-v3、tau-Bench 和 ACEBench-Agent 上平均得分 48.81，超过随机任务选择训练 120 步的最高分。
正文 · 原文
Authors:Sihan Yang, Weichen Bai, Ruichuan An, Yiming Cheng, Zhiyuan Feng, Tianyi Jiang, Renye Yan, Ning Ding, Yifan Zhang, Renrui Zhang, Yehui Tang, Wentao Zhang
Abstract:As executable environments for language agents scale, generating and verifying their rollouts becomes increasingly expensive. Under finite rollout budgets, this raises an important question: can data management at the environment level make agentic reinforcement learning (RL) more efficient by organizing, evaluating, and scheduling task instances together with the environments that determine their tools, states, transitions, and verification? To address this question, we introduce EnvSched (Environment-aware Scheduling), an online scheduler that learns from rollout history. EnvSched assigns scheduling value through two complementary modeling choices: exploitation prioritizes tasks likely to provide informative reward differences, while coverage spreads each batch across underobserved environments and interaction patterns. EnvSched organizes environment--task pairs as nodes in a dual-relation graph, with within-environment edges capturing task semantics and shared tool, state, and verifier structure, and cross-environment edges capturing abstract execution and reward topology. By propagating historical rollout statistics along both edge types, it predicts within-task reward variance. These predictions guide a scheduler that allocates each batch to tasks with high predicted reward variance and underobserved environments and tasks. Environment quotas and a similarity penalty further limit concentration and redundancy, yielding informative and diverse batches without additional pilot rollouts. In RL on Qwen3-8B, EnvSched achieves an average score of 48.81 across BFCL-v3, tau-Bench, and ACEBench-Agent after 40 training steps, exceeding the highest score attained by random task selection after 120 steps. It also improves downstream performance across model sizes at the same training budget.
| Subjects: | Machine Learning (cs.LG); Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.37609 [cs.LG] |
| (or arXiv:2610.37609v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.37609 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Sihan Yang [view email]
[v1]
Tue, 27 Oct 2026 17:38:52 UTC (2,163 KB)
来源：arXiv cs.CL · arxiv.org