arXiv cs.AI· Miao-Miao Yi, Zhen-Yu Cai, Y -M. Du, C. P. Sun·· 14 小时前AI 评分24
AI 知识蒸馏中的信息论可靠性-安全边界与有限对齐动力学
Information-Theoretic Reliability--Safety Bounds and Finite-Alignment Dynamics in AI Knowledge Distillation
AI 导读
研究证明了一条与模型形式无关的普适权衡不等式，其边界仅由单一参数 Kh 决定，Kh 量化隐藏层区分良性输入与危险输入的能力。基于 Boltzmann 机模型，可靠性-安全权衡曲线由危害判别能力 K 决定，整体回答倾向控制曲线上的工作点。在知识蒸馏中，教师模型的回答会诱导学生模型权衡参数的重整化，多代蒸馏的重整化群分析给出精确递推关系与不动点结构，可预测能力损失与稳定传递之间的临界转变。
正文 · 原文
Abstract:The reliability and safety of artificial intelligence (AI), together with their transfer and evolution through knowledge distillation, have attracted growing attention. We establish an information-theoretic bound on the reliability--safety trade-off and characterize the dynamics of this trade-off under knowledge distillation. We first prove a universal trade-off inequality that is independent of the specific model form. Its boundary is determined solely by a single parameter Kh, which quantifies the hidden layer's ability to distinguish benign from hazardous inputs. We then develop a canonical Boltzmann-machine model of AI response behavior and derive two exact relations between reliability and safety. The trade-off curve is determined by the hazard discrimination capacity K, while the overall answer tendency controls the operating point along the curve. These relations agree well with existing experimental data. In knowledge distillation, we find that teacher responses induce a renormalization of the student model's trade-off parameters. We further use a renormalization-group approach to study multigenerational distillation, deriving exact recursion relations and the associated renormalization-group-like flows for two representative direct intergenerational distillation protocols at arbitrary teacher--student alignment strength. These flows reveal fixed-point structures, long-term dynamics, and critical behavior, yielding testable predictions for the critical transition between capability loss and stable transmission.
| Subjects: | Statistical Mechanics (cond-mat.stat-mech); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.38651 [cond-mat.stat-mech] |
| (or arXiv:2610.38651v1 [cond-mat.stat-mech] for this version) | |
| https://doi.org/10.48550/arXiv.2610.38651 arXiv-issued DOI via DataCite |
Submission history
From: Miaomiao Yi [view email]
[v1]
Wed, 28 Oct 2026 11:04:21 UTC (4,112 KB)
来源：arXiv cs.AI · arxiv.org