arXiv cs.AI· Ruairidh M. Battleday, Kai Sandbrink, Jimi Cullen-Drohan, Zihan Yan, Timothy Muller, Clare Maguire, Ales Kubicek, Fraser Greenlee-Scott, Sukrit Sumant, Tri Dao, J\"urgen Schmidhuber, Michal Valko, Joshua Tenenbaum, Thomas L. Griffiths, Zeb Kurth-Nelson, James C. R. Whittington·· 14 小时前AI 评分55
DiG-bench：用 70 个规则未知的游戏评测智能体的发现能力
DiG-bench: Discovery in Games
AI 导读
研究团队发布 DiG-bench（Discovery in Games）基准，包含 70 个独立游戏，每个游戏以短字符串编码，其变换规则和关卡通关条件均需通过交互实验发现。基准设七档难度，最低档多个模型可常规解出，最高档对最强模型在智能体框架下构成挑战，全部 70 个游戏均被至少一名人类首次尝试即解出。其中 21 个游戏公开，其余保留私有用于安全评测。
正文 · 原文
Authors:Ruairidh M. Battleday, Kai Sandbrink, Jimi Cullen-Drohan, Zihan Yan, Timothy Muller, Clare Maguire, Ales Kubicek, Fraser Greenlee-Scott, Sukrit Sumant, Tri Dao, Jürgen Schmidhuber, Michal Valko, Joshua Tenenbaum, Thomas L. Griffiths, Zeb Kurth-Nelson, James C.R. Whittington
Abstract:Discovery---formulating novel generalizations---is a central part of the scientific process. Despite its importance, there is a gap in the current AI benchmark landscape, with few benchmarks directly probing the capacity for discovering new knowledge with experimentation in controlled environments where the objective is unknown. To address this gap, we release a new benchmark: DiG-bench (Discovery in Games). DiG-bench consists of a set of 70 independent games. Each game is encoded as a short string and has unique transformation rules that must be discovered through interaction and experimentation. The levels of the game present a series of challenges to test whether the rules have been discovered, where the win conditions for each level are also unknown. We provide games at seven tiers of difficulty for AI agents. The lowest tier is routinely solvable by multiple models, while the highest tier challenges the best models in agentic harnesses. All 70 games were solved by at least one human on first attempt. A subset of 21 games is released publicly, and the remainder is held private for secure evaluation.
| Subjects: | Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2608.12593 [cs.AI] |
| (or arXiv:2608.12593v2 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2608.12593 arXiv-issued DOI via DataCite |
Submission history
From: James Whittington [view email]
[v1]
Wed, 12 Aug 2026 21:06:06 UTC (820 KB)
[v2]
Tue, 27 Oct 2026 21:28:33 UTC (819 KB)
来源：arXiv cs.AI · arxiv.org