arXiv cs.AI· Joonhyeok Jeong, Jy-yong Sohn, Kibok Lee·· 12 小时前AI 评分20
重新审视长尾数据下监督对比学习的表征几何
Rethinking Representation Geometry in Supervised Contrastive Learning under Long-Tailed Data
AI 导读
研究提出理论框架，用于判断监督对比学习及其平衡变体在最优解处是否形成正单纯形几何，结果表明这些目标因依赖每类样本数而无法在最优处形成正单纯形。为此作者引入可施加于对比目标的结构平衡，并提出一种可证明在最优处形成正单纯形几何的完全平衡对比目标。理论与实验结果均显示，该方法得到更均衡的表征空间，并提升长尾识别与迁移学习任务的表现。
正文 · 原文
Abstract:Supervised contrastive learning and its balanced variants have demonstrated their effectiveness for representation learning under long-tailed distributions. However, although prior analyses suggest that their optimal representations may exhibit a regular simplex geometry, whether such a geometry is indeed exhibited remains insufficiently understood. In this work, we present a theoretical framework for these objectives that characterizes whether a regular simplex is formed at the optimum. By applying this to representative objectives, we show that these objectives fail to form a regular simplex at the optimum because they remain dependent on the number of samples per class. Motivated by this analysis, we introduce structural balancing that can be applied to contrastive objectives, and propose a fully balanced contrastive objective that provably forms a regular simplex geometry at its optimum. Both theoretical and empirical results demonstrate that the proposed method yields a more balanced representation space and improves performance on long-tailed recognition and transfer learning tasks.
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (stat.ML) |
| Cite as: | arXiv:2610.32039 [cs.LG] |
| (or arXiv:2610.32039v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.32039 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Kibok Lee [view email]
[v1]
Fri, 23 Oct 2026 01:30:14 UTC (692 KB)
来源：arXiv cs.AI · arxiv.org