arXiv cs.AI· Yue Zhang, Zun Wang, Han Lin, Yonatan Bitton, Idan Szpektor, Mohit Bansal·· 10 天前AI 评分41
SPATIALUNCERTAIN：VLM 能否识别不可靠空间观测（以及为何不能）？
Seeing Isn't Knowing: Do VLMs Know When Not to Answer Spatial Questions (and Why)?
AI 导读
研究团队提出 SPATIALUNCERTAIN 评测框架，考察视觉语言模型在遮挡导致证据缺失、透视导致证据误导两种失效模式下能否判断当前视角是否可靠。在八个开源与闭源 VLM 上测试发现，模型行为并不跟随视觉证据的可靠性：证据消失时不会更谨慎，透视冲突下判断更多跟随投影外观而非未变的物理 3D 关系。
正文 · 原文
Abstract:Spatial reasoning benchmarks typically evaluate whether vision-language models can derive the correct answer from a visual observation. Yet in real 3D environments, the observation itself may be unreliable: occlusion can remove task-relevant evidence, while perspective can make visible geometry misleading. Reliable spatial reasoning therefore requires more than answering a question correctly. A model must also assess whether its current observation provides sufficient and trustworthy evidence for that answer. We introduce SPATIALUNCERTAIN, a controlled evaluation framework for studying viewpoint-dependent observational uncertainty. We study two complementary failure modes: missing evidence caused by occlusion and misleading evidence caused by perspective. We further evaluate whether models can recognize when the current view is unreliable and identify a more informative observation. Across eight open- and closed-source vision-language models, we find that model behavior does not track the reliability of visual evidence. Models do not reliably become more cautious as evidence disappears, and under perspective conflict, their judgments increasingly follow projected appearance rather than the unchanged physical 3D relation. Internal analysis suggests a corresponding representational asymmetry: projected 2D relations are readily available, whereas the underlying physical 3D relation is barely decodable. Moreover, models that can identify an informative viewpoint when explicitly asked often fail to recognize when such an additional view is needed. These failures are not fully resolved by prompting or fine-tuning, and providing a better viewpoint is substantially more effective than adding depth information to the same misleading observation. Our results identify assessing the reliability of visual observations as a distinct and missing component of current spatial reasoning evaluation.
| Comments: | Website: this https URL |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Computation and Language (cs.CL) |
| Cite as: | arXiv:2605.30557 [cs.CV] |
| (or arXiv:2605.30557v2 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2605.30557 arXiv-issued DOI via DataCite |
Submission history
From: Yue Zhang [view email]
[v1]
Thu, 28 May 2026 20:44:47 UTC (13,319 KB)
[v2]
Tue, 6 Oct 2026 01:04:05 UTC (14,230 KB)
来源：arXiv cs.AI · arxiv.org