arXiv cs.AI· Hyesoo Hong, Min Soo Kim, Wonje Jeung, Sangyeon Yoon, Dongjae Jeon, Albert No·· 12 小时前AI 评分39
VLMs Trace Without Tracking：诊断视觉路径跟随中的失败
VLMs Trace Without Tracking: Diagnosing Failures in Visual Path Following
AI 导读
前沿视觉语言模型（VLMs）在受控条件下跟随简单连续线条时已表现出脆弱性，附近存在其他路径时会出现不稳定的路径保持失败。内部表征分析显示，判断正确路径延续所需的信息在视觉编码阶段就已减弱。模型规模扩展、额外推理和显式追踪指令均无法解决该问题，在更真实丰富的视觉场景中同样持续存在。
正文 · 原文
Abstract:Vision-language models (VLMs) perform strongly on multimodal benchmarks, yet their ability to reliably follow continuous visual structure remains unclear. We focus on path following, a basic operation that requires following a selected path through successive visual continuations. Before considering complex factors such as crossings or overlaps, we first test whether current VLMs can robustly follow a simple continuous line under controlled settings that remove such confounds. Across frontier models, we find that tracing is already fragile, with this vulnerability manifesting as unstable path preservation in the presence of other nearby paths. Analysis of internal representations shows that information needed to determine the correct path continuation becomes weaker, with this degradation already emerging in the visual encoding stage. Common remedies do not resolve this weakness: model scaling provides limited improvement, additional reasoning provides uneven gains at substantial cost, and explicit tracing instructions fail to stabilize performance. The same path-preservation failures and sensitivity to nearby paths persist in visually richer and more realistic settings, suggesting that current VLMs lack a robust mechanism for following a selected path.
| Comments: | Accepted at NeurIPS 2026 |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2605.15672 [cs.CV] |
| (or arXiv:2605.15672v2 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2605.15672 arXiv-issued DOI via DataCite |
Submission history
From: Hyesoo Hong [view email]
[v1]
Fri, 15 May 2026 06:48:53 UTC (13,615 KB)
[v2]
Fri, 23 Oct 2026 04:27:49 UTC (16,236 KB)
来源：arXiv cs.AI · arxiv.org