arXiv cs.AI· Ivan Karpukhin, Andrey Savchenko·· 23 小时前AI 评分22
AlignedHPO:基于代理引导的高效损失超参数优化框架
AlignedHPO: Efficient Surrogate-Guided Loss Hyperparameter Optimization
AI 导读
AlignedHPO 是一个利用梯度对齐与代理验证目标来调优可微损失超参数的框架,通过 Jacobian-vector 乘积降低超梯度计算的时间与内存开销,并用归一化对齐和梯度平滑解决梯度幅值歧义与随机不稳定性。在 generalized Focal Loss 与 ArcFace 上的实验显示,该方法在长尾分类和开集人脸识别任务中性能与隐式微分相当或更优,超梯度计算速度提升近 12 倍。
正文 · 原文
Abstract:Modern deep learning losses involve multiple hyperparameters, such as weights, scales, and margins, typically tuned manually or through expensive black-box optimization despite their differentiability. We propose AlignedHPO, an efficient framework for tuning differentiable loss hyperparameters using gradient alignment and surrogate validation objectives. Our method employs Jacobian-vector products to reduce hypergradient computation time and memory, while differentiable metric surrogates guide optimization toward task-specific evaluation criteria. We address gradient magnitude ambiguity and stochastic instability through normalized alignment and gradient smoothing. Experiments with generalized Focal Loss and ArcFace demonstrate effective adaptation across long-tailed classification and open-set face recognition. AlignedHPO achieves competitive or improved performance compared to implicit differentiation, with nearly 12x faster hypergradient computation. These results highlight the potential of efficient surrogate-guided loss hyperparameter optimization.
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.22658 [cs.LG] |
| (or arXiv:2611.22658v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2611.22658 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Ivan Karpukhin [view email]
[v1]
Tue, 17 Nov 2026 12:17:54 UTC (151 KB)
来源:arXiv cs.AI · arxiv.org