arXiv cs.CL· Jingpu Yang, Tianhao Li, Fengxian Ji, Yufeng Wang, Shuxin Zheng, Xiaoqing Zhang·· 2 天前AI 评分31
CAMR：面向大语言模型持续学习的兼容性感知记忆回放
CAMR: Compatibility-Aware Memory Replay for Continual Learning in Large Language Models
AI 导读
研究者提出兼容性感知记忆回放框架 CAMR，联合控制当前样本监督与历史回放，通过兼容性检查与学习式记忆门在三种动作间做在线决策。
正文 · 原文
Abstract:Continual learning requires large language models to acquire new knowledge and capabilities while retaining what they have learned. Fine-tuning and replay support acquisition and retention, but deciding whether incoming examples need supervision and which historical examples to revisit remains a challenge. We propose Compatibility-Aware Memory Replay (CAMR), an adaptive framework that jointly controls current-example supervision and historical replay. CAMR combines compatibility checks with a learned memory gate to guide an online policy over three actions: learning incoming examples with general replay, jointly reinforcing compatible related examples, and skipping direct supervision of verified mastered duplicates. The policy learns from acquisition, retention, and training-cost feedback under a bounded per-arrival replay budget. We evaluate CAMR with Qwen3-8B-Base on continual factual updating and cross-task and cross-domain learning. On WikiBigEdit, CAMR consistently leads evaluated methods in Original Match from 3K to 100K facts, reaching 93.66% after just 3K facts and surpassing Full-history Replay (LoRA) after 100K facts. It also achieves the highest final scores on all four selected TRACE tasks and maintains or improves earlier-task performance relative to post-acquisition levels in a sequence spanning financial question answering, mathematical reasoning, code generation, and summarization.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.36734 [cs.CL] |
| (or arXiv:2610.36734v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.36734 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Tianhao Li [view email]
[v1]
Tue, 27 Oct 2026 06:52:54 UTC (6,301 KB)
来源：arXiv cs.CL · arxiv.org