跳到正文
arXiv cs.AI· Jiaqi Wu, Yongqi Dong, Yijie Zhong, Maple Shao, Yide Tao, Linsen Cai, Weize Cai, Hai L. Vu, Alvaro Garcia Hernandez·· 23 小时前AI 评分18

基于冻结世界模型的残差引导轻量强化学习用于风险感知轨迹选择

Residual-Guided Lightweight Reinforcement Learning for Risk-Aware Trajectory Selection with a Frozen World Model

AI 导读

研究提出一种残差引导强化学习框架,在冻结世界模型与固定候选轨迹库下用PPO微调候选选择与有界轨迹残差的联合策略,微调仅更新16,106个参数,部署时只保留11,725参数的排序网络。

正文 · 原文

View PDF HTML (experimental)

Abstract:In automated driving, world models provide future scene predictions for evaluating candidate trajectories, but fixed selection rules may not capture scene-dependent trade-offs. We propose a residual-guided reinforcement learning (RL) framework for lightweight trajectory selection with a frozen world model and a fixed candidate bank. The framework uses proximal policy optimization (PPO) to fine-tune a joint policy over candidate selection and bounded trajectory residuals. A context-conditioned ranking network learns candidate preferences from feedback obtained by evaluating local perturbations of the selected trajectories. At inference, the selector combines learned preference and predicted risk ranks using fixed weights and returns the selected candidate trajectory without modification. Fine-tuning updates 16,106 parameters across the actor and critic, while the deployed selector retains only the 11,725-parameter ranking network, with the residual branch and critic used exclusively during training. In open-loop evaluation on the nuScenes validation set, the method reduces average Euclidean trajectory error (L2) by 3.99% relative to the RiskWorld model baseline without RL-based selection. Under matched budgets for feedback accesses and actor updates, residual exploration reduces L2 by 0.83% relative to the categorical PPO variant trained without residuals. The method also maintains lower L2 than RiskWorld when risk features are removed or shuffled.
Comments: 6 pages, 2 figures
Subjects: Artificial Intelligence (cs.AI); Emerging Technologies (cs.ET); Machine Learning (cs.LG); Robotics (cs.RO); Systems and Control (eess.SY)
Cite as: arXiv:2611.22998 [cs.AI]
  (or arXiv:2611.22998v1 [cs.AI] for this version)
  https://doi.org/10.48550/arXiv.2611.22998

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Yongqi Dong [view email]
[v1] Tue, 17 Nov 2026 16:12:55 UTC (17,982 KB)

来源:arXiv cs.AI · arxiv.org