arXiv cs.AI· Lute Lillo, Nick Cheney·· 3 天前AI 评分22
TeLAPA：超越单模型优化，在持续强化学习中保持可塑性
Beyond Single-Model Optimization: Preserving Plasticity in Continual Reinforcement Learning
AI 导读
研究提出持续强化学习框架 TeLAPA（Transfer-Enabled Latent-Aligned Policy Archives），将行为多样的策略邻域组织为按任务划分的存档，并维护共享潜在空间，使存档策略在非平稳漂移下仍可比较和复用。
正文 · 原文
Abstract:Continual reinforcement learning must balance retention with adaptation, yet many methods still rely on \emph{single-model preservation}, committing to one evolving policy as the main reusable solution across tasks. Even when a previously successful policy is retained, it may no longer provide a reliable starting point for rapid adaptation after interference, reflecting a form of \emph{loss of plasticity} that single-policy preservation cannot address. Inspired by quality-diversity methods, we introduce \textsc{TeLAPA} (Transfer-Enabled Latent-Aligned Policy Archives), a continual RL framework that organizes behaviorally diverse policy neighborhoods into per-task archives and maintains a shared latent space so that archived policies remain comparable and reusable under non-stationary drift. This perspective shifts continual RL from retaining isolated solutions to maintaining \emph{skill-aligned neighborhoods} with competent and behaviorally related policies that support future relearning. In our MiniGrid CL setting, \textsc{TeLAPA} learns more tasks successfully, recovers competence faster on revisited tasks after interference, and retains higher performance across a sequence of tasks. Our analyses show that source-optimal policies are often not transfer-optimal, even within a local competent neighborhood, and that effective reuse depends on retaining and selecting among multiple nearby alternatives rather than collapsing them to one representative. Together, these results reframe continual RL around reusable and competent policy neighborhoods, providing a route beyond single-model preservation toward more plastic lifelong agents.
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Neural and Evolutionary Computing (cs.NE) |
| Cite as: | arXiv:2604.15414 [cs.LG] |
| (or arXiv:2604.15414v3 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2604.15414 arXiv-issued DOI via DataCite |
Submission history
From: Lute Lillo [view email]
[v1]
Thu, 16 Apr 2026 17:06:54 UTC (4,491 KB)
[v2]
Tue, 9 Jun 2026 00:58:37 UTC (6,378 KB)
[v3]
Tue, 13 Oct 2026 17:46:41 UTC (6,378 KB)
来源：arXiv cs.AI · arxiv.org