arXiv cs.AI· Yanjie Dong·· 23 小时前AI 评分18
面向通信高效拜占庭鲁棒联邦学习的投影惩罚梯度跟踪框架
Projected Penalty-Gradient Tracking for Communication-Efficient Byzantine-Resilient Federated Learning
AI 导读
针对压缩上传会破坏惩罚方法的拜占庭鲁棒性,提出投影惩罚梯度跟踪框架:诚实节点上传惩罚梯度的压缩残差,服务器将重建状态投影到已知惩罚梯度球上,兼容任意收缩压缩器。
正文 · 原文
Abstract:We investigate communication-efficient federated learning over heterogeneous datasets in the presence of Byzantine adversaries capable of uploading arbitrary information. Penalty-based methods bound the influence of each worker through the bounded gradient of a penalty function; however, the bound is lost once the uploads are compressed, since a Byzantine worker can accumulate an unbounded state at the server. To retain the Byzantine resilience under compressed uploads, we propose a projected penalty-gradient tracking framework, where each honest worker uploads a compressed residual of its penalty gradient and the server projects each reconstructed state onto the known penalty-gradient ball. The framework accommodates any contractive compressor and yields two algorithms: fault-resilient proximal gradient with consistent sparsification (FRPG-CS), whose shared mask is protected by freshly drawn reserve rows, and lazily aggregated FRPG (LAG-FRPG), whose uploads are event-triggered. For smooth and strongly convex losses, we establish a finite-time bound on the expected objective gap, where the Byzantine error floor is independent of the compressor and the compression error decays faster than the optimization error. We also bound the expected number of lazy uploads and derive certificates that separate the finite-penalty model bias from the optimization error. Experiments on MNIST and Fashion-MNIST show that the proposed algorithms attain the highest accuracy among twelve methods under the Min-Max attack, whereas a compressed-momentum benchmark and a local-update benchmark remain preferable in attack-free accuracy and total traffic, respectively.
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.22929 [cs.LG] |
| (or arXiv:2611.22929v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2611.22929 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Yanjie Dong [view email]
[v1]
Tue, 17 Nov 2026 15:28:03 UTC (116 KB)
来源:arXiv cs.AI · arxiv.org