arXiv cs.AI· Zhuoran Liang, Francisco Cruz·· 10 小时前AI 评分18
Encoder BPA：用潜在状态编码实现交互式强化学习中的广域持久化建议
Broad-Persistent Advising in Interactive Reinforcement Learning via Latent State Encoding
AI 导读
研究提出 Encoder BPA，用训练好的编码器将状态序列映射为潜在表示，通过潜在比较检索并复用建议，并结合 top-(k) 聚合与一致性条件提升复用稳定性。实验显示，该方法在训练早期相比多种 Deep IntRL 方法提升了学习效率与建议利用率，在复杂和部分可观测环境中尤为明显。该工作已被 ACRA 2026 接收为口头报告。
正文 · 原文
Abstract:Deep Interactive Reinforcement Learning (Deep IntRL) accelerates reinforcement learning training by incorporating external advice. However, conventional Deep IntRL uses each piece of advice only in the current step and discards it after a single interaction. Persistent Advising and Broad-Persistent Advising (BPA) extend this process by storing advice in persistent memory and reusing it in similar states through state generalisation based on K-means clustering. This research proposes Encoder BPA, which employs a latent encoder to retrieve and reuse advice. In this framework, a trained encoder is used to map a state sequence into a latent representation at every time step. Advice can be retrieved through latent comparison, together with a top-(k) aggregation and a consistency condition to improve reuse stability. Experimental results show that Encoder BPA improves learning efficiency and advice utilisation at an early stage of training compared with different Deep IntRL approaches, especially in complex and partially observable environments.
| Comments: | 10 pages, 5 figures, 3 tables, Accepted for oral presentation at the Australasian Conference on Robotics and Automation (ACRA 2026) |
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.13712 [cs.AI] |
| (or arXiv:2611.13712v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2611.13712 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Zhuoran Liang [view email]
[v1]
Tue, 10 Nov 2026 08:06:06 UTC (1,104 KB)
来源：arXiv cs.AI · arxiv.org