arXiv cs.CL· Zhaoxi Zhang, Hao Liang, Linxuan Wu, Yiming Wu, Yifu Yuan, Kun Liang, Chenming Tang, Meiyi Qiang, Wentao Zhang, Weikang Li·· 12 小时前AI 评分38
SpatialScout：VLM 智能体能否主动探索并构建 3D 场景地图？
SpatialScout: Can VLM Agents Actively Explore and Map 3D Scenes?
AI 导读
研究者推出 SpatialScout 基准，用于评测 VLM 智能体在主动物体级 3D 建图上的表现，包含 80 个室内场景、3140 个目标物体。智能体需在有限交互预算下控制相机探索，并估计物体中心坐标，评测同时衡量定位精度与目标覆盖率。最强模型 Claude Opus 5.5 平均重叠得分仅 41.71%，IoU 超过 0.5 的目标占 47.23%，超过 0.9 的仅 3.25%。
正文 · 原文
Abstract:Spatial intelligence requires agents to actively acquire visual evidence and integrate observations into a coherent representation of the three-dimensional world. Yet spatial question answering offers limited evidence of whether models can construct accurate metric maps through autonomous exploration. We introduce SpatialScout, a benchmark for evaluating vision-language model (VLM) agents on active object-level 3D mapping. SpatialScout comprises 80 diverse indoor scenes containing 3,140 target objects. Given an object inventory, RGB observations, camera calibration, and camera poses, agents control a camera to explore each scene and estimate object centers in a shared coordinate frame. The task requires coordinating viewpoint selection, cross-view association, geometric estimation, and map revision under bounded interaction budgets. Evaluation jointly measures localization accuracy and target coverage through object-scale overlap scores that penalize missing predictions. Experiments with frontier VLMs reveal substantial limitations: the strongest evaluated model, Claude Opus 5.5, achieves a mean overlap score of 41.71%, with 47.23% of targets exceeding an intersection-over-union (IoU) threshold of 0.5 and only 3.25% exceeding 0.9. These results reveal a substantial gap between recovering coarse object layouts and constructing complete, precise metric maps. SpatialScout provides a quantitative testbed for developing agents that build reliable 3D spatial representations through active perception. Code is available at this https URL.
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.26659 [cs.CV] |
| (or arXiv:2610.26659v2 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2610.26659 arXiv-issued DOI via DataCite |
Submission history
From: Hao Liang [view email]
[v1]
Mon, 19 Oct 2026 16:17:48 UTC (2,425 KB)
[v2]
Tue, 20 Oct 2026 02:49:48 UTC (2,426 KB)
来源：arXiv cs.CL · arxiv.org