arXiv cs.AI· Changxu Ni, Feiran You, Yongan Guo, Hongyang Du, Kang Wei, Le Liang, Yiyang Ni, Jun Li·· 13 小时前AI 评分24
CORMA：面向可泛化多约束离线安全强化学习的约束路由决策 Mamba
CORMA: Constraint-Routed Decision Mamba for Generalizable Multi-Constraint Offline Safe Reinforcement Learning
AI 导读
CORMA 是一个约束路由决策 Mamba 框架，用于可泛化的多约束离线安全强化学习，将策略推理拆分为约束感知的 RTG 估计与自适应动作生成。它用约束专属专家建模可行 RTG 分布，并通过路由引导的专家乘积机制为 Mamba 动作解码器提供安全感知引导。在 OSRL 基准上，CORMA 取得更优的奖励-安全权衡，相比 DT 类方法推理延迟降低 37%，对未见安全预算组合的零样本泛化更强。
正文 · 原文
Abstract:Offline safe reinforcement learning (RL) learns safety-aware policies from static datasets while satisfying safety constraints during deployment. Motivated by the strong trajectorymodeling capabilities of Transformers, recent Decision Transformer (DT)-based methods formulate offline safe RL as conditional sequence modeling by conditioning action generation on return-to-go (RTG) and cost-to-go (CTG) tokens, thereby enabling adaptation to varying safety budgets. However, existing DT-based methods are developed for single-constraint tasks. For tasks with multiple constraints, generalizing learned policies to unseen combinations of safety budgets remains challenging due to the conflicting requirements imposed by heterogeneous constraints. Moreover, the quadratic complexity of Transformer architectures significantly increases the computational overhead in multi-constraint tasks, as representing additional constraints requires more CTG tokens. Recently, Mamba has emerged as an efficient sequence-modeling backbone, offering linear computational complexity and strong generalization capabilities. In this paper, we propose CORMA, a constraint-routed decision Mamba framework for generalizable multi-constraint offline safe RL. CORMA decomposes policy inference into constraint-aware RTG estimation and adaptive action generation, where constraint-specific experts model feasible RTG distributions and a router-guided product-of-experts mechanism fuses these distributions to provide safety-aware guidance for a Mamba-based action decoder. A three-stage optimization strategy is designed to improve training stability. Extensive experiments on OSRL benchmarks demonstrate that CORMA achieves superior reward-safety trade-offs, reduces inference latency by 37% compared with DT-based methods, and exhibits stronger zero-shot generalization to unseen safety-budget combinations than offline safe RL baselines.
| Comments: | 30pages, 6 figures, conference |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.34208 [cs.LG] |
| (or arXiv:2610.34208v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.34208 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Changxu Ni [view email]
[v1]
Sun, 25 Oct 2026 10:54:34 UTC (1,261 KB)
来源：arXiv cs.AI · arxiv.org