arXiv cs.CL· Ada Defne T\"ur, Gaurav Kamath, Joyce Chai, Siva Reddy, Benno Krojer·· 12 小时前AI 评分31
新研究揭示 VLM 与人类如何映射全新视觉指称：NVRD 数据集与基准发布
Would you still call this Dax? Novel Visual References in VLMs and Humans
AI 导读
研究者构建 Novel Visual References Dataset（NVRD），含 19,176 张图像、覆盖 90 个视觉概念，每个概念最多 20 个逐步扰动版本，用于探测泛化能力。
正文 · 原文
Abstract:Vision-language models (VLMs), like human learners, are frequently exposed to new visual concepts, but how they map novel visual references to language after exposure remains largely underexplored, particularly when those references contradict prior knowledge from pre-training. To study this, we present the Novel Visual References Dataset (NVRD): 19,176 images spanning 90 visual concepts across different levels of visual novelty, each with up to 20 increasingly perturbed versions of the original object to probe generalization. Unlike prior work on visual augmentations of familiar concepts, NVRD comprises entirely novel, open-ended stimuli constructed from scratch, mirroring how humans encounter genuinely new concepts. We evaluate 3 open- and 2 closed-source models alongside 2,400 human judgments for direct human-model comparison, and find that (i) models struggle to acquire novel concepts in-context when they contradict prior knowledge, and (ii) while models and humans show correlated sensitivity to visual perturbations, models significantly overgeneralize, extending learned labels to stimuli that humans reject. We contribute NVRD as a corpus and benchmark for research on visual concept learning in both humans and machines.
| Comments: | EMNLP Main Conference 2026 |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL) |
| Cite as: | arXiv:2606.05409 [cs.CV] |
| (or arXiv:2606.05409v4 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2606.05409 arXiv-issued DOI via DataCite |
Submission history
From: Ada Tur [view email]
[v1]
Wed, 3 Jun 2026 20:23:12 UTC (9,189 KB)
[v2]
Mon, 8 Jun 2026 17:17:01 UTC (9,190 KB)
[v3]
Wed, 17 Jun 2026 15:52:34 UTC (9,190 KB)
[v4]
Fri, 16 Oct 2026 15:49:10 UTC (9,190 KB)
来源：arXiv cs.CL · arxiv.org