arXiv cs.AI· Kushagra Tiwary, David Mayo, Nikhil Behari, Xiangzhou Sun, Abdulrahman Alabdulkareem, Isaac Galatzer-Levy, Boris Katz, Brian Cheung·· 12 小时前AI 评分38
ArtifactArena：用物理世界中的建造能力评测前沿模型
ArtifactArena: Evaluating Models by What They Build in the Physical World
AI 导读
ArtifactArena 是一个开放式评测平台，让模型在物理仿真竞技场中完成软硬件协同设计挑战——工程化制造可运行的机器人并参赛。平台通过文本描述、物理模拟器反馈和游戏数据三种 harness 对模型进行零样本、验证器引导优化和开放式物理设计能力测试，并以模型产物间头对头锦标赛的 Elo 排名进行基准比较。该框架和锦标赛基础设施开放社区持续提交，旨在构建一个不饱和的活体测试平台。
正文 · 原文
Abstract:To evaluate the frontier, we must measure models not by what they say, but by what they can engineer and build in grounded physical environments. We introduce ArtifactArena, an open-ended platform where models face a physically grounded hardware-software co-design challenge: engineering fully functional robots to compete in a simulated arena. We evaluate a frontier model's zero-shot, verifier guided refinement, and open-ended physical design capabilities through three harnesses that refine their bot artifacts based on text descriptions, physics simulator feedback, and gameplay data. We benchmark these capabilities with an Elo ranking of frontier models derived from head-to-head tournaments between their artifacts. By releasing this framework and tournament infrastructure for ongoing community submissions, we establish a living, non-saturating testbed to continuously measure the expanding limits of open-ended intelligence in the physical world. Please visit this https URL for more information.
| Subjects: | Robotics (cs.RO); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.06511 [cs.RO] |
| (or arXiv:2610.06511v2 [cs.RO] for this version) | |
| https://doi.org/10.48550/arXiv.2610.06511 arXiv-issued DOI via DataCite |
Submission history
From: Kushagra Tiwary [view email]
[v1]
Mon, 5 Oct 2026 15:28:48 UTC (7,125 KB)
[v2]
Thu, 22 Oct 2026 18:05:54 UTC (7,125 KB)
来源：arXiv cs.AI · arxiv.org