跳到正文
arXiv cs.AI· Xuefei Zhang, Ruoqi Wang, Lei Cui·· 10 小时前AI 评分18

Angular Conditional Distillation:解决分类器几何失配的知识蒸馏新方法

Angular Conditional Distillation: Overcoming Classifier Geometry Mismatch

AI 导读

研究者提出 Angular Conditional Distillation(ACD),通过 Angular Coordinate Interface(ACI)以角坐标重建学生分类器、抑制幅度与偏置影响,并用 Scale-Invariant Conditional Distillation(SICD)标准化输出、只迁移条件性非目标关系。

正文 · 原文

View PDF HTML (experimental)

Abstract:Classifier outputs provide a fundamental and univer- sally available knowledge interface for knowledge distil- lation (KD). However, standard logit objectives entangle transferable semantic relations with architecture-dependent parameterization artifacts, causing a classifier geometry mismatch that makes direct knowledge transfer unreliable. To address this, we propose Angular Conditional Distilla- tion (ACD). First, the Angular Coordinate Interface (ACI) reconstructs the student classifier using angular coordi- nates, suppressing magnitude and bias effects to provide a geometry-compatible space. Then, Scale-Invariant Condi- tional Distillation (SICD) standardizes outputs and exclu- sively transfers conditional non-target relations, filtering out untransferable absolute confidence. Together, ACI de- fines where to transfer, while SICD defines what to transfer. Extensive experiments show that ACD achieves state-of-the- art performance over existing KD methods. Moreover, as a universal drop-in replacement, it consistently improves existing distillation pipelines while requiring no feature adapters or additional inference-time cost. Code is avail- able at this https URL.
Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
Cite as: arXiv:2611.23941 [cs.CV]
  (or arXiv:2611.23941v1 [cs.CV] for this version)
  https://doi.org/10.48550/arXiv.2611.23941

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Xuefei Zhang [view email]
[v1] Wed, 18 Nov 2026 06:28:17 UTC (2,345 KB)

来源:arXiv cs.AI · arxiv.org