arXiv cs.AI· Shenghui Li, Li Ju, Thiemo Voigt·· 10 小时前AI 评分31
HiLo：用压缩梯度同步替代 LoRA 传输，加速分布式 on-policy 后训练
Train High, Ship Low: Efficient LoRA Synchronization for Distributed On-Policy Post-Training
AI 导读
HiLo 提出用压缩梯度分发与本地优化器重放替代 LoRA 适配器参数传输，在带宽受限的分布式 on-policy 后训练中保持学习质量接近全量同步，传输字节仅为适配器的 1.6%，端到端更新最高加速 2.2 倍。配套的 Spar 是预算感知梯度压缩器，结合谱截断、自适应秩分配与误差反馈，将训练秩与通信成本解耦。
正文 · 原文
Abstract:Transmitting LoRA adapters to rollout workers can bottleneck distributed on-policy post-training over bandwidth-constrained networks. Lowering the training rank may limit the capacity a workload needs, while adaptive optimizer updates are difficult to compress. Rollout workers can instead reconstruct updates from gradients by maintaining replicated optimizer states. We introduce HiLo, a compressor-agnostic framework that replaces parameter transmission with compressed gradient dissemination and local optimizer replay. By applying the same decoded gradients, the learner and workers maintain consistent policy states at the same version under reproducible execution. We further develop Spar, a budget-aware gradient compressor that combines spectral truncation, adaptive rank allocation and error feedback to decouple training rank from communication cost. Experiments across on-policy distillation and reward-guided post-training show that HiLo with Spar maintains learning quality close to full synchronization while transmitting as little as 1.6% of the adapter bytes, and accelerates end-to-end updates by up to $2.2\times$ under bandwidth constraints.
| Comments: | 19 pages, 7 figures |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.13306 [cs.LG] |
| (or arXiv:2611.13306v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2611.13306 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Shenghui Li [view email]
[v1]
Mon, 9 Nov 2026 22:18:34 UTC (1,654 KB)
来源：arXiv cs.AI · arxiv.org