arXiv cs.CL· Zixiang Xu, Zirui Song, Wanrong Zheng, Chiyu Zhang, Yue Huang, Yuyang Peng, Kexin Zhang, Jinkun Chen, Xiuying Chen, Xiangliang Zhang, Yan Liu, Laurent Itti, Frank Shyu, Xiaoqiang Luo, Sandeep Pandey, Xi Liu, Luke Simon, Xiyang Hu, Yue Zhao·· 2 天前AI 评分31
Grid-Bench：LLM 能否预测离散世界的变化？
Grid-Bench: Can LLMs Predict How Discrete Worlds Change?
AI 导读
研究者推出 Grid-Bench，一个可执行基准，用二、三、四维网格表示离散世界，让模型在给定初始网格、显式规则和文本事件历史后预测完整最终网格。评估六款 LLM 显示，随着历史变长，完整状态准确率急剧下降，单事件表现相近的模型出现明显分化。参考执行使每个预测值可核验，从而在空间操作、历史长度和工作负载间直接比较。
正文 · 原文
Authors:Zixiang Xu, Zirui Song, Wanrong Zheng, Chiyu Zhang, Yue Huang, Yuyang Peng, Kexin Zhang, Jinkun Chen, Xiuying Chen, Xiangliang Zhang, Yan Liu, Laurent Itti, Frank Shyu, Xiaoqiang Luo, Sandeep Pandey, Xi Liu, Luke Simon, Xiyang Hu, Yue Zhao
Abstract:Spatial instructions describe how to move objects, transform regions, or modify their surroundings. When several operations are applied in sequence, each acts on a configuration shaped by earlier changes. How accurately can large language models (LLMs) predict the resulting spatial state? We introduce Grid-Bench, an executable benchmark in which eight families of operations transform discrete worlds represented as two-, three-, and four-dimensional grids. Given an initial grid, explicit rules, and an event history expressed as text, a model predicts the complete final grid. Reference execution makes every predicted value checkable, supporting comparisons across spatial operations, history lengths, and workloads. Evaluating six LLMs shows that complete-state accuracy falls sharply as histories grow, with substantial divergence among models that perform similarly on single events. Models with similar exact-match accuracy can nevertheless differ substantially in the number of incorrect cells on shared instances. Even over long histories, some models recover many correct changes beyond what copying the initial state would yield. Grid-Bench makes both complete recovery and these differences in returned states directly measurable across a range of spatial conditions.
| Comments: | 39 pages, 15 figures, 15 tables, including appendices. Project website: this https URL ; Code: this https URL ; Data: this https URL |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.22312 [cs.CL] |
| (or arXiv:2610.22312v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.22312 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Zixiang Xu [view email]
[v1]
Thu, 15 Oct 2026 17:50:57 UTC (737 KB)
来源：arXiv cs.CL · arxiv.org