跳到正文
arXiv cs.AI· Jiachun Li, Zhuoran Jin, Kang Liu, Jun Zhao·· 4 小时前AI 评分31

从感知到模拟:揭示并增强 MLLM 的心理空间动态能力

From Perception to Simulation: Unveiling and Empowering Mental Spatial Dynamics in MLLMs

AI 导读

研究提出 MentalGrid 基准,用于评测多模态大模型在无中间视觉反馈下模拟空间状态演变的能力,评测显示现有模型空间模拟能力明显不足。团队用 16k 专项空间推理样本的三阶段课程学习框架训练出 SpatialWalker,平均较基座模型提升 33.8%,在空间模拟上超过 Gemini-3-Flash,并在域外基准上保持 4.9% 的平均增益。

正文 · 原文

View PDF HTML (experimental)

Abstract:Spatial reasoning is a fundamental component of world understanding for Multimodal Large Language Models (MLLMs). While existing research has advanced the perception of spatial relations in static scenes, the domain of dynamic spatial simulation remains relatively unexplored. To bridge this gap, we introduce MentalGrid, a benchmark focused on spatial reasoning tasks that require models to simulate evolving spatial states without intermediate visual feedback. Extensive evaluation on our benchmark reveals a significant deficiency in the spatial simulation capabilities of existing models. Specifically, we find that performance is constrained by perceptual limitations in simple simulation scenarios and hindered by inadequate multi-step reasoning as complexity increases. To address these challenges, we propose a three-stage curriculum learning framework utilizing 16k specialized spatial reasoning samples. This approach yields SpatialWalker, which achieves a 33.8% average improvement over its base model, surpassing proprietary models like Gemini-3-Flash in spatial simulation. Furthermore, SpatialWalker demonstrates robust generalization, maintaining a 4.9% average gain on out-of-domain benchmarks, demonstrating the pivotal role of spatial simulation as a fundamental building block for broader spatial intelligence.
Comments: Findings of EMNLP 2026
Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
Cite as: arXiv:2611.22541 [cs.CV]
  (or arXiv:2611.22541v1 [cs.CV] for this version)
  https://doi.org/10.48550/arXiv.2611.22541

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Jiachun Li [view email]
[v1] Tue, 17 Nov 2026 10:31:56 UTC (1,236 KB)

来源:arXiv cs.AI · arxiv.org