跳到正文
arXiv cs.AI· Minjae Kwon, Amir Moeini, Shangtong Zhang, Lu Feng·· 4 小时前AI 评分31

Latent Q-Barrier Shielding:面向安全上下文强化学习的部署期动作过滤方法

Latent Q-Barrier Shielding for Safe In-Context Reinforcement Learning

AI 导读

研究者提出 Latent Q-Barrier shield,在部署前学习上下文表示、潜在动力学与集成成本评论家,部署时无需参数更新即可依据剩余预算和预测未来成本过滤或软重加权候选动作。

正文 · 原文

View PDF HTML (experimental)

Abstract:Safe in-context reinforcement learning (ICRL) adapts online from interaction history without test-time parameter updates while controlling episode cost under a safety budget. Under out-of-distribution (OOD) deployment shifts, pretraining-only safe ICRL can give poor reward-safety tradeoffs because the remaining budget affects behavior only through frozen policy conditioning, not an explicit action-level check against predicted future cost. We propose a latent Q-Barrier shield that learns a context representation, latent dynamics, and an ensemble cost critic before deployment. Without parameter updates, the shield infers context from history and filters or softly reweights candidate actions using the remaining budget and predicted future cost. We prove a conditional one-step barrier-margin bound over realized candidate sets, with explicit Bellman and latent-prediction error terms. Across five safe ICRL benchmarks, the shield improves deployment-time reward-safety tradeoffs over a strong safe-ICRL baseline: after a short context window, it achieves higher return in four of five benchmarks while matching or lowering average episode cost in all five. Code is available at this https URL.
Comments: Accepted at NeurIPS 2026. Camera-ready version
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
Cite as: arXiv:2605.25267 [cs.LG]
  (or arXiv:2605.25267v2 [cs.LG] for this version)
  https://doi.org/10.48550/arXiv.2605.25267

arXiv-issued DOI via DataCite

Submission history

From: Minjae Kwon [view email]
[v1] Sun, 24 May 2026 21:45:28 UTC (355 KB)
[v2] Mon, 16 Nov 2026 21:56:38 UTC (337 KB)

来源:arXiv cs.AI · arxiv.org