Know-Thy-Not：双编码器视觉语言模型的无训练定向否定

Know-Thy-Not: Training-Free Targeted Negation in Dual-Encoder Vision-Language Models

AI 导读

双编码器视觉语言模型在否定理解上系统性失败，对"没有狗的猫"和"有狗的猫"打分几乎相同，该问题源于单嵌入框架的固有缺陷。研究者提出无训练方法 Know-Thy-Not（KTN），在图像侧引入拒绝维度直接惩罚被否定概念，并发布首个需要真正拒绝才能通过的基准 COCO-Confusion。

正文 · 原文

View PDF HTML (experimental)

Abstract:Dual-encoder vision-language models systematically fail at negation, scoring ''a cat without a dog'' nearly identically to ''a cat with a dog.'' We show that this failure is inherent to the single-embedding framework. Under contrastive training, single-embedding methods can at best ignore the negated concept (dilution) but never penalize its presence (rejection). We propose Know-Thy-Not (KTN), a training-free method that introduces an image-side rejection dimension to penalize the negated concept directly in the visual representation. We also introduce COCO-Confusion, the first benchmark requiring genuine rejection to succeed. Across 9 backbones, KTN improves COCO-Confusion mAP by up to +25.6 over vanilla VLMs and +9.6 over the strongest existing method.

Comments:	9 pages
Subjects:	Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
Cite as:	arXiv:2610.17440 [cs.CV]
	(or arXiv:2610.17440v1 [cs.CV] for this version)
	https://doi.org/10.48550/arXiv.2610.17440 arXiv-issued DOI via DataCite (pending registration)

Submission history

From: DoYoung Kim [view email]
[v1] Mon, 12 Oct 2026 21:58:20 UTC (4,575 KB)

来源：arXiv cs.AI · arxiv.org