arXiv cs.AI· Anissa Alloula, Charles Jones, Zuzanna Wakefield-Skorniewska, Francesco Quinzan, Bart{\l}omiej Papie\.z·· 12 小时前AI 评分29
子群平衡何时重要？用潜在可分性预测性能敏感性
When does Subgroup Balance Matter? Predicting Performance Sensitivity from Latent Separability
AI 导读
研究提出“潜在分离假说”：微调模型对子群数据比例的依赖程度，由预训练模型潜在空间中子群之间的分离程度决定。该假说经理论分析并在四个视觉与语言数据集上得到实证验证，正则化实验提供了其因果作用的初步证据。基于潜在子群分离度的定量分析可指导基础模型微调时的数据采集与平衡决策。
正文 · 原文
Abstract:Unequal representation of demographic groups in training data poses challenges to model generalisation across populations. Standard practice assumes that balancing subgroup representation always optimises performance. However, recent empirical results contradict this assumption: in some cases, imbalanced data distributions actually improve subgroup performance, and in others, subgroup performance is unaffected by the absence of an entire subgroup during training. We propose the latent separation hypothesis, which states that a fine-tuned model's dependence on subgroup representation is determined by the degree of separation between subgroups in the latent space of the pre-trained model. We formalise this hypothesis, provide theoretical analysis, validate it empirically across four vision and language datasets, and present preliminary evidence of a causal role through regularisation experiments. Finally, we show how quantitative analysis of latent subgroup separation can inform practical data collection and balancing decisions when fine-tuning foundation models.
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2512.09496 [cs.LG] |
| (or arXiv:2512.09496v2 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2512.09496 arXiv-issued DOI via DataCite |
Submission history
From: Anissa Alloula [view email]
[v1]
Wed, 10 Dec 2025 10:19:48 UTC (3,705 KB)
[v2]
Tue, 20 Oct 2026 15:31:37 UTC (13,902 KB)
来源：arXiv cs.AI · arxiv.org