arXiv cs.CL· Mahdieh Mallahnezhad, Masoud Jafaripour, Mohamadali Tofigh, Sheila Schoepp, Osmar R. Zaiane·· 3 天前AI 评分20
多模态大模型空间推理的数据中心化理解综述
Towards a Data-Centric Understanding of Spatial Reasoning in Multimodal Large Language Models
AI 导读
一篇综述系统梳理了面向空间推理任务微调预训练多模态大语言模型（MLLMs）的方法，涵盖监督学习到强化学习，重点关注所用数据集。研究将相关工作分为数据集设计、数据集构建与评估三类，并归纳出规则模板、LLM 提示、程序化生成和人在回路四种固定空间标注构建方法。评估方面存在评分方法、容差指标和基准类型的显著差异，未来方向包括统一动态标注、高效空间推理、评估标准化、扩展空间范围与以人为中心的空间交互。
正文 · 原文
Abstract:Multimodal Large Language Models (MLLMs) have been widely applied in embodied agents and robotics for vision tasks. However, MLLMs suffer from limitations in spatial reasoning and perception. Post-training techniques have shown promise for improving these capabilities; yet, there are limited data-driven surveys of such approaches. In this survey, we consider approaches to fine-tuning pretrained MLLMs for spatial reasoning tasks, ranging from supervised to reinforcement learning, with an emphasis on the datasets used. We investigate themes of dataset construction, data modality and annotation, and evaluation of the resulting models. Based on these themes, we categorize research into three groups: dataset design, dataset construction and evaluation. We find four main methods for constructing fixed spatial annotations: rules and templates, LLM prompting, procedural generation, and human-in-the-loop. We also observe substantial variation in evaluation approaches, including different scoring methods, tolerance metrics, and benchmark types. Finally, we identify opportunities for future research in unified dynamic annotation, efficient spatial reasoning, standardization of evaluation, expanded spatial range, and human-centered spatial interactions.
| Comments: | 29 pages, 3 figures, 2 tables, Under review at Artificial Intelligence Review |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.19108 [cs.CV] |
| (or arXiv:2610.19108v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2610.19108 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Mahdieh Mallahnezhad [view email]
[v1]
Tue, 13 Oct 2026 19:42:30 UTC (2,739 KB)
来源：arXiv cs.CL · arxiv.org