arXiv cs.AI· Qiming Shi, Yikun Li, Beidi Luan, Rui Sun, Yan Ma, Jing Li, Daxin Jiang, Zuo Bai·· 13 小时前AI 评分39
AutoVend：LLM 智能体长周期自主售货运营基准测试
AutoVend: Benchmarking LLM Agents on Long-Horizon Autonomous Vending Operations
AI 导读
AutoVend 基准让单个 LLM 智能体通过网页运营界面管理 20 台自动售货机，覆盖五类场地，模拟运行 180 天。在五个受测 LLM 智能体中，有四个未能超过基于规则的库存控制基线；按每美元推理成本回报排名时，模型顺序与单看回报时不同。运营分析显示，库存与执行成本会侵蚀单项指标的增益，轨迹分析则暴露出智能体在落地可行动作、管理库存承诺和将反馈转化为有效策略修订上的困难。
正文 · 原文
Abstract:Large language model (LLM) agents are extending agent-mediated commerce on the Web from shopping for consumers to operating businesses for sellers. In online-to-offline (O2O) commerce, decisions are issued through Web-based platforms but realized through costly physical execution. Evaluating such agents requires going beyond individual transactions to ask whether decisions made online turn into sustained economic returns. We introduce AutoVend, a benchmark in which one LLM agent manages 20 vending machines across five site types through a Web-based operating interface for 180 simulated days. Its data-grounded simulator combines a shared warehouse, procurement lead times, paid replenishment, inventory expiration, and recurring operating costs. We evaluate five LLM agents against a rule-based inventory-control baseline and diagnose their site coordination, execution, and performance over time. Four of five evaluated LLM agents fall short of this baseline, and returns per dollar of inference cost rank models differently from returns alone. Operational analyses show that gains in individual metrics do not consistently yield stronger economic growth, as inventory and execution costs erode these gains. Trajectory analysis further reveals difficulties in grounding feasible actions, managing inventory commitments, and converting feedback into effective policy revisions.
| Subjects: | Artificial Intelligence (cs.AI); Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.36980 [cs.AI] |
| (or arXiv:2610.36980v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.36980 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Zuo Bai [view email]
[v1]
Tue, 27 Oct 2026 10:33:06 UTC (596 KB)
来源：arXiv cs.AI · arxiv.org