arXiv cs.CL· Jun Wang, Xiaohao Xu, Xiaonan Huang·· 12 小时前AI 评分33
TouchSafeBench：评估视觉语言模型碰撞定位能力的物理基准
Probing Collision Grounding in Vision-Language Models for Safe Human-Robot Collaboration
AI 导读
研究者提出 TouchSafeBench，一个基于 Habitat 3.0 构建的物理基准，含 2,940 个室内共处模拟片段，用于评估视觉语言模型的碰撞定位能力。在三个前沿或机器人导向 VLM 与九种视觉表示上，最佳平均 Macro-F1 仍低于 50%，显式深度信息不会自动转化为机器人本体碰撞证据，且机器人-场景接触始终比人体接触风险更难判断。
正文 · 原文
Abstract:Safe human--robot collaboration requires more than visual description: a monitor must determine whether the robot body is safely separated, already colliding with the scene or a person, or about to collide. We call this capability collision grounding: binding visual observations to robot body geometry, camera viewpoint, scene layout, human proximity, and temporal motion in order to infer present and imminent contact. We introduce TouchSafeBench, a physics-grounded benchmark for evaluating collision grounding in vision-language models (VLMs). Built in Habitat~3.0, TouchSafeBench contains 2,940 simulated indoor co-presence episodes across social navigation and social rearrangement, with synchronized multi-view RGB-D observations, top-down trajectory maps, calibrated camera metadata, and simulator-derived contact labels. We study two deployment-facing tasks: classifying the current safety state and warning about imminent collision before contact. Across three frontier or robotics-oriented VLMs and nine visual representations, current models remain far from reliable: the best average Macro-F1 stays below 50\%, explicit depth is not automatically transformed into robot-body collision evidence, and robot--scene contact is consistently harder than human-contact risk. TouchSafeBench reveals a central limitation of embodied VLMs: visual fluency does not imply physical accountability. Reliable robot safety monitors will need representations that explicitly bind viewpoint, robot morphology, metric geometry, and future collision. The benchmark is available at this https URL .
| Comments: | Accepted by 10th Conference on Robot Learning (CoRL 2026), Austin TX, USA |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Robotics (cs.RO) |
| Cite as: | arXiv:2605.31196 [cs.CV] |
| (or arXiv:2605.31196v2 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2605.31196 arXiv-issued DOI via DataCite |
Submission history
From: Xiaohao Xu [view email]
[v1]
Fri, 29 May 2026 12:04:38 UTC (7,088 KB)
[v2]
Fri, 23 Oct 2026 10:43:59 UTC (6,034 KB)
来源：arXiv cs.CL · arxiv.org