arXiv cs.CL· Shangxi Wu, Xiaorong Yan, Xiaojun Meng, Yuxiang Zhang, Jiansheng Wei·· 14 小时前AI 评分31
ECHO：用熵控制与动态反馈预防 LLM 监督微调中的灾难性遗忘
Entropic Control for Homeostatic Optimization: Pre-empting Catastrophic Forgetting via Dynamic Feedback
AI 导读
研究者提出闭环控制框架 ECHO，通过分解 KL 散度将输出熵作为监测遗忘风险的在线代理，无需昂贵的参考模型推理。ECHO 用弹性控制器按比例控制律动态调节回放数据的梯度权重，以偏离预训练熵平衡的程度作为误差信号。在 0.6B 至 9B 参数架构上的实验显示，ECHO 位于并多次扩展了可塑性-稳定性前沿，且对数据混合比例变化保持高鲁棒性。
正文 · 原文
Abstract:Supervised Fine-Tuning (SFT) of LLMs suffers from catastrophic forgetting, and prevailing mitigation strategies struggle to reconcile the plasticity-stability dilemma. To address this, we propose Entropic Control for Homeostatic Optimization (ECHO), a closed-loop control framework. By decomposing the KL divergence, we identify output entropy as a dominant observable component of KL variation. This enables us to leverage entropy as a computationally efficient online proxy to monitor forgetting risks in real-time, bypassing expensive reference-model inference. Consequently, ECHO employs an elastic controller that dynamically modulates the gradient weight of replay data via a proportional control law, using deviations from the pre-trained entropic equilibrium as an error signal. Empirical evaluations across 0.6B to 9B parameter architectures demonstrate that ECHO lies on, and in several comparisons extends, the observed plasticity-stability frontier and maintains high operational robustness against varying data mixing ratios.
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.34182 [cs.LG] |
| (or arXiv:2610.34182v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.34182 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Shangxi Wu [view email]
[v1]
Sun, 25 Oct 2026 09:34:29 UTC (370 KB)
来源：arXiv cs.CL · arxiv.org