arXiv cs.CL· Sajjad Pakdamansavoji, Mohammad Reza Rezaei, Ilija Subasic, Pedro Garzon, Radu Paradovschi, Zhao Chen·· 11 小时前AI 评分31
OLGA：面向持续语言模型训练的无标签在线梯度匹配方法
Online Label-Free Gradient Matching for Continual Language Model Training
AI 导读
研究者提出 OLGA，一种无标签梯度匹配方法，用模型自身的下一 token 预测生成梯度匹配目标，从而摆脱对真实标签的依赖。在 phi-1.5 上的五个文本分类基准中，仅用五个样本即比最强选择基线提升 10 个百分点准确率；在线数据流场景下比最强选择式持续学习基线高 6.6 个 stream-AUC 百分点。
正文 · 原文
Abstract:Language models in customer-facing products must be continually updated, since the data they serve shifts over time. As retraining on the full accumulated data is impractical, each update should instead run on a small set of examples carrying the training signal of all data seen so far. The strongest current methods build this set by gradient matching, which synthesizes examples whose training gradients match those of the full data. However, they require ground-truth labels, run once on a fixed snapshot rather than on a growing stream, and are slow on large data. We introduce \textbf{OLGA}, which lifts these limitations. It distills unlabeled data by drawing the gradient matching targets from the model's own next-token predictions. Across five text classification benchmarks with phi-1.5, it improves accuracy by 10 percentage points over the strongest selection baseline using as few as five examples. OLGA also distills online data streams by keeping a running rehearsal memory of synthetic examples, which is re-distilled together with each new batch as it arrives. In the online setting, it outperforms the strongest selection-based continual baseline by 6.6 stream-AUC percentage points. Finally, our method scales to large data by pairing distillation with selection in a hybrid pipeline, letting selection handle the mild stage of compression and distillation the extreme one. It cuts runtime by 30\% versus distillation alone at similar accuracy, and beats the best selection-only pipeline by up to 12.6 percentage points.
| Comments: | 18 pages, 5 figures, 12 tables |
| Subjects: | Artificial Intelligence (cs.AI); Computation and Language (cs.CL) |
| ACM classes: | I.2.7; I.2.6 |
| Cite as: | arXiv:2611.03496 [cs.AI] |
| (or arXiv:2611.03496v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2611.03496 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Sajjad Pakdamansavoji [view email]
[v1]
Mon, 2 Nov 2026 17:37:36 UTC (416 KB)
来源：arXiv cs.CL · arxiv.org