arXiv cs.AI· Yijun Zhou, Minghao Shao, Ke Zhang, Renkai Yuan, Hongbo Yuan, Bin Hu, Bozhao Qian, Hongxia Shen, Yanhong Liu, Yuting Chen, Yuntao Shi, Zhongxuan Xu, Shuo Mao, Haijia Zhou, Zhendong Li, Tao Zhang, Chenggang Li, Yuxuan Wang, Zinan Zheng, Calvin Zhang, Jia Li, Heng Chang, Jie Feng·· 13 小时前AI 评分35
UnityBench-Mini3D：评估 LLM 智能体在 Unity 中进行 3D 游戏开发的能力
UnityBench-Mini3D: Evaluating LLM Agents for 3D Game Development in Unity
AI 导读
UnityBench-Mini3D 是一个包含 77 个项目级 Unity 任务的基准，按七类专家定义的技能和三个难度层级组织，用 1,178 条人工编写的留出里程碑测试在引擎内评估场景结构、组件配置与运行时行为。
正文 · 原文
Authors:Yijun Zhou, Minghao Shao, Ke Zhang, Renkai Yuan, Hongbo Yuan, Bin Hu, Bozhao Qian, Hongxia Shen, Yanhong Liu, Yuting Chen, Yuntao Shi, Zhongxuan Xu, Shuo Mao, Haijia Zhou, Zhendong Li, Tao Zhang, Chenggang Li, Yuxuan Wang, Zinan Zheng, Calvin Zhang, Jia Li, Heng Chang, Jie Feng
Abstract:Game development stresses coding agents in ways general software engineering does not: correctness depends on how engine APIs, serialized scene state, and runtime behavior interact, not on source code alone. Existing benchmarks mainly target platforms such as Godot and heavily rely on LLM judges or coarse task-level outcomes. We present UnityBench-Mini3D, a benchmark of 77 project-level Unity tasks organized around seven expert-defined skill categories rather than tutorials or genres, with three difficulty tiers. Each task pairs a complete Unity project with natural-language requirements and is evaluated using 1,178 held-out, human-authored milestone tests covering scene structure, component configuration, and runtime behavior. This in-engine evaluation provides deterministic partial credit, localizes failures, and distinguishes partial implementations from complete solutions without relying on subjective judges. Across 21 models and 9 coding harnesses, the average pass rate reaches 80.9% on easy tasks but falls to 35.2% on hard tasks, with just 57.7% overall. UnityBench-Mini3D provides a reproducible, diagnostic testbed distinguishing systems across difficulty levels.
| Subjects: | Software Engineering (cs.SE); Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.35980 [cs.SE] |
| (or arXiv:2610.35980v1 [cs.SE] for this version) | |
| https://doi.org/10.48550/arXiv.2610.35980 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Minghao Shao [view email]
[v1]
Mon, 26 Oct 2026 10:19:14 UTC (9,287 KB)
来源：arXiv cs.AI · arxiv.org