arXiv cs.AI· Loan Bernat (LAAS-GEPETTO), Matthieu Grard (LAAS-RAP), Ariane Herbulot (LAAS-RAP), Florent Lamiraux (LAAS-GEPETTO)·· 6 天前AI 评分38
MAGMA-Bench：评估动态任务与执行失败下的长时程具身智能体
MAGMA-Bench: Evaluating Long-Horizon Embodied Agents under Dynamic Tasks and Execution Failures
AI 导读
研究者推出 MAGMA-Bench，一个架构无关的基准，用于评估动态多机器人环境中的持续任务推理，覆盖递增任务时程与执行失败、任务中断、任务更新等在线扰动。评测显示最强模型在相当比例的干净回合中仍失败，在线扰动下成功率进一步下降；监督适配能大幅提升小模型，但在更长时程和多机器人任务上表现仍明显偏低。团队同时发布基准、261,646 条任务级训练样本及架构无关的专家轨迹生成流程。
正文 · 原文
Abstract:Long-horizon embodied agents must maintain task-relevant information, coordinate heterogeneous robots, and adapt their decisions as execution unfolds. Existing benchmarks investigate dynamic task execution and embodied multi-agent coordination, but provide limited evaluation of their intersection. We introduce MAGMA-Bench, an architecture-agnostic benchmark for persistent task reasoning in dynamic multi-robot environments. MAGMA-Bench evaluates agents across increasing task horizons and controlled online perturbations, including execution failures, task interruptions, and mission updates, with scenarios requiring coordination across robots with heterogeneous capabilities. It uses a structured skill-level interface to separate task-level reasoning from perception and low-level control. We evaluate locally deployable open-weight planners, multi-agent state-tracking strategies, and frontier models. Even the strongest evaluated models fail on a substantial fraction of clean episodes, and their success rates decrease further under online perturbations. Supervised adaptation greatly improves a small model, but performance remains substantially lower on longer and multi-robot missions. We release the benchmark, 261,646 task-level training examples, and an architecture-agnostic expert-trace generation pipeline.
| Subjects: | Artificial Intelligence (cs.AI); Robotics (cs.RO) |
| Cite as: | arXiv:2610.13534 [cs.AI] |
| (or arXiv:2610.13534v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.13534 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Loan BERNAT [view email] [via CCSD proxy]
[v1]
Fri, 9 Oct 2026 12:54:11 UTC (3,244 KB)
来源：arXiv cs.AI · arxiv.org