arXiv cs.AI· Haiyun Guo, Hongyu Guo, Xiangzhao Hao, Jiarui Guo, Jinqiao Wang, Tat-Seng Chua·· 2 天前AI 评分22
UniFGVC：基于属性感知多模态检索的通用免训练少样本细粒度视觉分类
UniFGVC: Universal Training-Free Few-Shot Fine-Grained Vision Classification via Attribute-Aware Multimodal Retrieval
AI 导读
UniFGVC 是一个免训练的少样本细粒度视觉分类框架，将分类任务重构为基于预定义类别模板的多模态检索问题。其核心 CDV-Captioner 模块借助 MLLM 生成结构化文本描述，结合链式推理与视觉参考定位相似类别的判别性视觉属性，抑制模型幻觉。在 12 个 FGVC 数据集上，该方法不仅超越现有少样本 CLIP 方法，还超过多个全监督 MLLM 方法。
正文 · 原文
Abstract:Few-shot fine-grained visual classification (FGVC) aims to discriminate between subtly different categories with very limited data. While Multimodal Large Language Models (MLLM) exhibit strong capabilities in both fine-grained visual understanding and open-world category recognition, directly fine-tuning them on few-shot FGVC tasks typically leads to overfitting on sparse samples, which erodes their valuable open-world knowledge and compromises generalization. To fundamentally overcome this limitation, we present UniFGVC, a universal training-free framework that shifts the paradigm from parameter fine-tuning to training-free retrieval-based inference. UniFGVC reframes few-shot FGVC as a multimodal retrieval problem using predefined category templates. Its core is the Category-Discriminative Visual Captioner (CDV-Captioner), an attribute-aware module designed to leverage MLLM for generating structured textual descriptions that explicitly pinpoint discriminative visual attributes between similar classes. By integrating chain-of-thought reasoning and visual references, CDV-Captioner mitigates hallucination and maximizes the discriminative power of textual captions, thereby converting images into attribute-enriched image-description pairs. Subsequently, utilizing arbitrary pre-trained visual and textual encoders in a plug-and-play manner, UniFGVC extracts features from both modalities and computes their similarity with all category templates for recognition. Extensive experiments on 12 FGVC datasets demonstrate UniFGVC's universal compatibility and consistent superiority. It not only outperforms existing few-shot CLIP-based methods but also surpasses several fully-supervised MLLM-based approaches, confirming the general advantage of the proposed retrieval paradigm.
| Comments: | Accepted by IEEE TMM |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2508.04136 [cs.CV] |
| (or arXiv:2508.04136v4 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2508.04136 arXiv-issued DOI via DataCite |
Submission history
From: Hongyu Guo [view email]
[v1]
Wed, 6 Aug 2025 07:02:39 UTC (1,272 KB)
[v2]
Fri, 23 Jan 2026 10:16:53 UTC (2,022 KB)
[v3]
Tue, 3 Feb 2026 09:30:40 UTC (2,022 KB)
[v4]
Thu, 15 Oct 2026 15:30:05 UTC (5,252 KB)
来源：arXiv cs.AI · arxiv.org