arXiv cs.AI· Yiming Yan, Hongyang Wang, Wenbo Chen, Xuan Wang, Jingguo Liu, Xian Li, Chen Qi, Leo Fan, Siwei Li, Qianli Wu, Yichuan Li, Ruochen Wen, Qian Qu, Zheshen Wang, Benoit Dumoulin·· 14 小时前AI 评分52
Sandman：在共享 Kubernetes 上为智能体强化学习构建并运营沙箱
Sandman: Building and Operating Sandboxes for Agentic Reinforcement Learning on Shared Kubernetes
AI 导读
Amazon 团队提出沙箱服务 Sandman，用于智能体工具执行与评测，在共享 Kubernetes 上支撑多个生产级智能体强化学习负载。每个沙箱是一个 Kubernetes pod，镜像中挂载一个小型守护进程，文件与进程在调用间持续保留，直到训练框架释放环境。
正文 · 原文
Authors:Yiming Yan, Hongyang Wang, Wenbo Chen, Xuan Wang, Jingguo Liu, Xian Li, Chen Qi, Leo Fan, Siwei Li, Qianli Wu, Yichuan Li, Ruochen Wen, Qian Qu, Zheshen Wang, Benoit Dumoulin
Abstract:Agentic reinforcement learning (RL) often requires repeated tool calls in task-specific environments. A training job may create thousands of these environments in bursts while sharing a cluster with other jobs. We present Sandman, a sandbox service built and operated at Amazon for agent tool execution and evaluation. It serves multiple production agentic RL workloads on shared Kubernetes. A training framework supplies a task image and uses commands and file operations in the returned sandbox. Files and processes persist across calls until the framework releases the environment. Sandman manages creation, readiness and cleanup. Each sandbox is a Kubernetes pod with a small daemon mounted into the unmodified image. In a scale test, Sandman held 11,858 concurrent tiny sandboxes. Four production SWE-bench Verified training campaigns created 3.1 million sandboxes in October 2026. We report the design trade-offs and operational lessons from handling creation bursts, preparing task environments, reclaiming abandoned sandboxes, and running evaluation under controlled conditions. We will open-source Sandman's daemon, provider and pool.
| Comments: | 42 pages, 6 figures, 53 tables |
| Subjects: | Distributed, Parallel, and Cluster Computing (cs.DC); Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.39711 [cs.DC] |
| (or arXiv:2610.39711v1 [cs.DC] for this version) | |
| https://doi.org/10.48550/arXiv.2610.39711 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Yiming Yan [view email]
[v1]
Thu, 29 Oct 2026 09:45:57 UTC (214 KB)
来源：arXiv cs.AI · arxiv.org