arXiv cs.AI· Zilin Du, Haoxin Li, Jianfei Yu, Boyang Li·· 12 小时前AI 评分26
POBF：面向视觉定位的数据稀缺训练数据合成与筛选框架
Paint Outside the Box: Synthesizing and Selecting Training Data for Visual Grounding
AI 导读
POBF（Paint Outside the Box and Filter）通过框外图像修复合成训练图像，解决以往工作中的标签错位问题，并用结合难度分数与过拟合分数、由惩罚项平衡的筛选方案挑选最有效训练数据。在四个基准数据集上，POBF 相比仅用真实数据的方法平均提升 5.83%，准确率领先主流基线 2.29%-3.85%。该框架在不同生成模型、训练数据规模和模型架构下均验证了鲁棒性与泛化性。
正文 · 原文
Abstract:Visual grounding aims to localize the image regions based on a textual query. Given the difficulty of large-scale data curation, we investigate how to effectively learn visual grounding under data-scarce settings in this paper. To address the data scarcity, we propose a novel framework, POBF (Paint Outside the Box and Filter). POBF synthesizes images by inpainting outside the box, tackling a label misalignment issue encountered in previous works. Furthermore, POBF leverages an innovative filtering scheme to select the most effective training data. This scheme combines a hardness score and an overfitting score, balanced by a penalty term. Extensive experiments across four benchmark datasets demonstrate that POBF consistently improves performance, achieving an average gain of 5.83\% over the real-data-only method and outperforming leading baselines by 2.29\%-3.85\% in accuracy. Additionally, we validate the robustness and generalizability of POBF across various generative models, training data sizes, and model architectures.
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2412.00684 [cs.CV] |
| (or arXiv:2412.00684v3 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2412.00684 arXiv-issued DOI via DataCite |
Submission history
From: Zilin Du [view email]
[v1]
Sun, 1 Dec 2024 05:47:59 UTC (2,785 KB)
[v2]
Sun, 20 Apr 2025 10:32:37 UTC (8,267 KB)
[v3]
Fri, 16 Oct 2026 09:26:02 UTC (3,261 KB)
来源：arXiv cs.AI · arxiv.org