arXiv cs.AI· Nishanth Anand, Doina Precup·· 12 小时前AI 评分31
用参数化与非参数化表示分离永久与瞬态价值函数，提升持续强化学习
Permanent and Transient Representations for Continual Reinforcement Learning
AI 导读
该研究提出让长期学习依赖参数化表示、瞬态适应依赖非参数化表示，以缓解持续强化学习中的稳定性-可塑性权衡。作者为复杂任务中的瞬态价值函数引入一种新的非参数化近似方法，使系统支持在线学习。在图像类任务和 Craftax-Classic 上，该方法优于竞争基线，论文已被 NeurIPS 2026 接收。
正文 · 原文
Abstract:Continual Reinforcement Learning (CRL) agents struggle to adapt to new situations while retaining prior knowledge, leading to a stability-plasticity trade-off. One approach to solving this problem is by using complementary learning systems: one for slow, long-term learning and another for quick, transient adaptation. Anand & Precup (2023) instantiated this idea using permanent and transient value functions, but used the same features in both systems. Intuitively, long-term learning should rely more on parametric representations for broad generalization, while transient learning would be best served by non-parametric representations for situation-specific adaptation. In this paper, we explore this idea both theoretically and empirically. By introducing a novel non-parametric approximation for estimating the transient value function in complex tasks, we demonstrate that our method enables online learning and outperforms competitive baselines on image-based tasks and Craftax-Classic, underscoring the effectiveness of our system-level decomposition.
| Comments: | Accepted at the 40th Conference on Neural Information Processing Systems (NeurIPS 2026) |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.07327 [cs.LG] |
| (or arXiv:2611.07327v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2611.07327 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Nishanth Anand [view email]
[v1]
Thu, 5 Nov 2026 02:11:47 UTC (17,635 KB)
来源：arXiv cs.AI · arxiv.org