arXiv cs.AI· Qianpu Chen, Derya Soydaner, Rob Saunders·· 11 小时前AI 评分39
视觉模型中的空想性错视：用 FacesInThings 数据集探测人脸选择性
Faces in Things, Faces in Nothing: Probing Face Selectivity in Vision Models with Pareidolia
AI 导读
研究用 FacesInThings 数据集测试六种视觉模型（从纯视觉 ViT 到 CLIP 及生成式视觉语言模型），发现模型对错觉人脸的反应与人类不同：人脸检测器很少在这些区域返回人脸，而 CLIP ViT-L/14 和两个生成模型在类别被描述为"人脸种类"时把约一半区域标为 Human，标注者则读作动物或卡通脸。
正文 · 原文
Abstract:An electrical outlet can look like a face without depicting a person. Face pareidolia is thus a revealing probe of how vision models turn ambiguous visual evidence into category judgments. When a model calls such an object Human, does that answer depend on its face-like arrangement? Using the FacesInThings dataset, we test six vision models, from a vision-only ViT to CLIP and generative vision-language models, with face detectors as a reference. Each face-like region is paired with three controls from the same photograph that lack the arrangement: a background crop, the region with its tiles shuffled, and its average color. A Human answer driven by the arrangement should appear less often on these controls. We also vary how the categories are described. Three findings emerge. First, models respond to illusory faces, but not as people do: face detectors rarely return a face for these regions, yet CLIP ViT-L/14 and two generative models label roughly half of them Human when the categories are described as kinds of face. Annotators, by contrast, read these faces as non-human, such as animal or cartoon faces. For these models, a face is a human face by default. Second, similar Human rates hide opposite behavior. LLaVA-1.5 answers Human less often once the arrangement is removed; Qwen2-VL calls every average-color patch Human and describes a person in each; CLIP ViT-B/32 calls shuffled regions Human more often than real faces; SmolVLM rarely answers Human, but mostly for face-like regions. Third, rewording the categories changes whether Human answers favor the face at all, removing this preference in LLaVA-1.5 and creating it in CLIP. A Human answer is not evidence of a face: vision models can keep giving it after the face is gone.
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2603.03989 [cs.CV] |
| (or arXiv:2603.03989v4 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2603.03989 arXiv-issued DOI via DataCite |
Submission history
From: Qianpu Chen [view email]
[v1]
Wed, 4 Mar 2026 12:33:36 UTC (3,687 KB)
[v2]
Tue, 21 Jul 2026 13:11:18 UTC (5,496 KB)
[v3]
Wed, 22 Jul 2026 15:07:20 UTC (2,412 KB)
[v4]
Wed, 4 Nov 2026 13:17:16 UTC (10,212 KB)
来源：arXiv cs.AI · arxiv.org