arXiv cs.AI· Xuefei Zhang, Ruoqi Wang, Lei Cui·· 10 小时前AI 评分18
Angular Conditional Distillation:解决分类器几何失配的知识蒸馏新方法
Angular Conditional Distillation: Overcoming Classifier Geometry Mismatch
AI 导读
研究者提出 Angular Conditional Distillation(ACD),通过 Angular Coordinate Interface(ACI)以角坐标重建学生分类器、抑制幅度与偏置影响,并用 Scale-Invariant Conditional Distillation(SICD)标准化输出、只迁移条件性非目标关系。
正文 · 原文
Abstract:Classifier outputs provide a fundamental and univer- sally available knowledge interface for knowledge distil- lation (KD). However, standard logit objectives entangle transferable semantic relations with architecture-dependent parameterization artifacts, causing a classifier geometry mismatch that makes direct knowledge transfer unreliable. To address this, we propose Angular Conditional Distilla- tion (ACD). First, the Angular Coordinate Interface (ACI) reconstructs the student classifier using angular coordi- nates, suppressing magnitude and bias effects to provide a geometry-compatible space. Then, Scale-Invariant Condi- tional Distillation (SICD) standardizes outputs and exclu- sively transfers conditional non-target relations, filtering out untransferable absolute confidence. Together, ACI de- fines where to transfer, while SICD defines what to transfer. Extensive experiments show that ACD achieves state-of-the- art performance over existing KD methods. Moreover, as a universal drop-in replacement, it consistently improves existing distillation pipelines while requiring no feature adapters or additional inference-time cost. Code is avail- able at this https URL.
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.23941 [cs.CV] |
| (or arXiv:2611.23941v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2611.23941 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Xuefei Zhang [view email]
[v1]
Wed, 18 Nov 2026 06:28:17 UTC (2,345 KB)
来源:arXiv cs.AI · arxiv.org