arXiv cs.CL· Tianchen Guo, Tengyue Zhang, Chen Liu, Ling Chen, Xin Yu·· 13 小时前AI 评分31
SATE-Bench：MLLMs 能否通过主动外视角探索构建以人为中心的空间理解？
Humans in Space: Can MLLMs Construct Human-Centric Spatial Understanding via Active Exocentric Exploration?
AI 导读
研究团队提出 SATE-Bench，包含 316 个真实场景和 2525 条多轮 QA 对，用于评测 MLLMs 的视觉自我意识与主动外视角探索能力。评测显示，当前 SOTA MLLMs 会在证据不足时过早终止探索却报告高置信度，且难以处理细粒度 3D 几何任务。用 GRPO 微调奖励视角寻求虽能提升主动探索，却因过度探索和多轮多视角上下文融合失败而降低整体准确率。
正文 · 原文
Abstract:While Multimodal Large Language Models (MLLMs) excel at zero-shot reasoning, their spatial capabilities are primarily evaluated through static, passive observation. In complex, human-centric scenes, occlusion often hides crucial geometric evidence, and passive benchmarks force models to guess rather than recognize their own visual uncertainty. To bridge this gap, we introduce SATE-Bench, a novel interactive benchmark comprising 316 real-world scenes and 2,525 multi-turn QA pairs to evaluate visual self-awareness and active exocentric view-exploration. It combines a Passive multi-view control with an Active exploration setting, where models start from a severely occluded view and must iteratively query adjacent viewpoints until they gather enough evidence to fully ``sate'' the question, scored by a step-penalized efficiency reward. Evaluating state-of-the-art MLLMs reveals that they terminate exploration prematurely while reporting high confidence, and struggle with fine-grained 3D geometric tasks; matched-view controls further attribute the Active--Passive gap primarily to insufficient evidence acquisition. Moreover, fine-tuning with Group Relative Policy Optimization (GRPO) shows that rewarding view-seeking increases active exploration but lowers overall accuracy, as models over-explore and fail to fuse expanding multi-view contexts across turns. These findings indicate that scaling passive multi-view perception does not intrinsically yield active spatial reasoning, highlighting the need for models that know where to look, how to integrate what they see, and when to stop. Code and data are available at this https URL
| Comments: | Winter Conference on Applications of Computer Vision (WACV) 2027. 33 pages, 14 figures, and 23 tables |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.34149 [cs.CV] |
| (or arXiv:2610.34149v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2610.34149 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Tianchen Guo [view email]
[v1]
Sun, 25 Oct 2026 08:35:20 UTC (3,280 KB)
来源：arXiv cs.CL · arxiv.org