arXiv cs.AI· Zeyun Deng, Jasorsi Ghosh, Fiona Xie, Kai Cheng, Yuzhe Lu, Katia Sycara, Joseph Campbell·· 13 小时前AI 评分20
基于能量的迁移方法提升强化学习样本效率
Energy-Based Transfer for Reinforcement Learning
AI 导读
研究者提出一种基于能量的迁移学习方法，利用分布外检测让预训练教师策略只在自身训练分布内的状态上提供引导，避免新任务差异过大时误导探索。该方法在单任务和多任务设置下均提升了样本效率与性能，论文已被 CoLLAs 2026 接收。
正文 · 原文
Abstract:Reinforcement learning remains sample inefficient, particularly when agents must adapt over time to related but distinct tasks, as in continual learning settings. Efficiency can be improved by transferring knowledge from a pretrained teacher policy to guide exploration in new but related tasks. However, if the new task sufficiently differs from the teacher's training task, the transferred guidance may be sub-optimal and bias exploration toward low-reward behaviors. We propose an energy-based transfer learning method that uses out-of-distribution detection to selectively issue guidance, enabling the teacher to intervene only in states within its training distribution. We theoretically link energy scores to state-visitation patterns under on-policy reinforcement learning. Empirically, our method improves sample efficiency and performance across both single-task and multi-task settings.
| Comments: | Accepted to CoLLAs 2026 |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2506.16590 [cs.LG] |
| (or arXiv:2506.16590v2 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2506.16590 arXiv-issued DOI via DataCite |
Submission history
From: Fiona Xie [view email]
[v1]
Thu, 19 Jun 2025 20:25:52 UTC (8,198 KB)
[v2]
Tue, 20 Oct 2026 23:13:36 UTC (16,172 KB)
来源：arXiv cs.AI · arxiv.org