arXiv cs.CL· Ge Qu, Jinyang Li, Xiaohan Xu, Xiaolong Li, Bowen Qin, Nan Huo, Ziwei Tang, Chenhao Ma, Reynold Cheng·· 13 小时前AI 评分35
BIRD-RL：面向有状态数据中心环境的智能体强化学习框架
BIRD-RL: Scaling Agentic Reinforcement Learning over Stateful Data-Centric Environments
AI 导读
BIRD-RL 提出四阶段智能体强化学习课程，交替增强基础推理与轨迹自适应在线策略探索，并配套轨迹级持久执行基础设施，实现数据库副本安全复用且保持轨迹隔离，将空间 RL 效率提升 12.8 倍，无需 Kubernetes。
正文 · 原文
Abstract:While agentic reinforcement learning (RL) has emerged as a promising paradigm for training small language models (SLMs), existing frameworks are predominantly tool-centric and struggle with data-centric environments. Specifically, applying tool-centric frameworks to stateful tasks like SQL debugging introduces severe spatial inefficiencies since proper environment isolation typically requires a separate database container per rollout. Naively reusing databases without tracking evolving states across multi-turn interactions causes trajectory contamination and cross-trajectory interference, which severely destabilizes policy updates. To overcome these bottlenecks, we introduce BIRD-RL, a novel four-stage agentic RL curriculum that alternately enhances foundational agent reasoning and trajectory-adaptive on-policy exploration to circumvent the instability inherently associated with direct end-to-end learning. Crucially, to support this curriculum, BIRD-RL is driven by a Trajectory-scoped Persistent Execution Infrastructure, which enables safe database replica reuse while preserving trajectory isolation. This design improves spatial RL efficiency by 12.8x, successfully enabling scalable agentic RL for complex data-centric tasks without needs of Kubernettes. Empirical evaluations on the BIRD-CRITIC-SQLite benchmark demonstrate that our 7B and 14B models achieve Success Rates of 44.40% and 48.00%, respectively, showing competitive performance against strong proprietary model-based agents such as Claude-Opus-4.6 Agent (48.20%). Furthermore, BIRD-RL successfully trains unified SLMs capable of both advanced SQL debugging and generation, highlighting its broader potential for multi-objective optimization across heterogeneous, real-world data-centric tasks.
| Comments: | Accepted to NeurIPS 2026 Main Track |
| Subjects: | Machine Learning (cs.LG); Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.29426 [cs.LG] |
| (or arXiv:2610.29426v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.29426 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Ge Qu [view email]
[v1]
Wed, 21 Oct 2026 08:50:39 UTC (1,364 KB)
来源：arXiv cs.CL · arxiv.org