arXiv cs.MA· Ningpei Ding, Zai Wen Feng·· 12 小时前AI 评分55
SlowLab：在为期一年、不可撤销的实验上评测自主 AI 智能体
SlowLab: Benchmarking Autonomous AI Agents on a Year of Experiments They Cannot Undo
AI 导读
SlowLab 是一个让智能体在四个模拟温室隔间中执行为期一年实验活动的基准，场地作物、设施与价格均对智能体隐藏，最终由私有评估器在智能体未见过的天气下为其推荐的气候策略打分。
正文
Abstract:AI agents are increasingly proposed as autonomous experimenters, yet the benchmarks that test them mostly offer cheap experiments that answer at once. Experiments in physical facilities are different: they take months, occupy scarce equipment and cannot be undone. We present SlowLab, a benchmark in which an agent runs a one-year campaign in four simulated greenhouse compartments on a site whose crop, facility and prices are hidden. The agent decides when to plant which climate policy, whether to stop a crop and which measurements to read while crops grow, and at the end recommends one policy, which a private evaluator scores in weather the agent never saw. In a preregistered study on 48 unseen sites, three low-cost language-model agents ended near a standard grower setting and clearly below a Bayesian-optimisation baseline that starts from that setting. They ran the facility as two synchronous batches, so measurements taken during the season never changed what they planted, and these measurements improved no method's result. Letting other methods re-read the evidence a campaign had collected did not yield significantly better recommendations, which points to where the models start and which experiments they try. Code is released under the MIT license.
| Comments: | 16 pages (9 main text), 7 figures, 5 tables. Code: this https URL |
| Subjects: | Artificial Intelligence (cs.AI); Multiagent Systems (cs.MA) |
| Cite as: | arXiv:2610.34644 [cs.AI] |
| (or arXiv:2610.34644v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.34644 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Ning Pei Ding [view email]
[v1]
Mon, 26 Oct 2026 00:09:19 UTC (522 KB)
来源：arXiv cs.MA · arxiv.org