一颗红心触发递归：VLM 视觉意义与交流能力研究

View PDF HTML (experimental)

Abstract:Do vision-language models (VLMs) share our ability to communicate visually? We share preliminary findings, starting from a surprising observation: simple visual symbols like a red heart can strongly influence model behavior on seemingly unrelated tasks, such as steering models towards recursive solutions on simple coding problems. Notably, this steering effect is largely implicit, as models' direct interpretations and thinking traces about visual symbols diverge substantially from their observed behavior. In visual communication between models, this leads to a generation--interpretation gap: while models tend to send shape primitives that they (and humans) find intuitive, these visual cues often fail to steer other models. In further communication games involving abstract concepts and stories, models can creatively use visuals to transmit meaning to one another, although some communications remain non-intuitive to humans. These findings reveal that VLMs might be capable of rich, creative, and visually-grounded communication, but their interpretation and production of visual meaning diverges from human conventions in surprising ways. Together this work highlights both the promise and challenges of visual communication between humans and VLMs.

Subjects:	Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Computer Vision and Pattern Recognition (cs.CV)
Cite as:	arXiv:2610.36449 [cs.AI]
	(or arXiv:2610.36449v1 [cs.AI] for this version)
	https://doi.org/10.48550/arXiv.2610.36449 arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Alexa Tartaglini [view email]
[v1] Tue, 27 Oct 2026 00:51:17 UTC (14,761 KB)