跳到正文
arXiv cs.AI· Jia Hu, Yang Chang, Panqu Wang, Zhijun Fu, Chang Liu, Changjian Yu, Jintao Lai·· 23 小时前AI 评分34

闭环交互式世界模型让自动驾驶场景更真实:行为与视觉渲染耦合

Making the World React Realistically: A Closed-Loop Interactive World Model for Autonomous Driving

AI 导读

研究者提出一种闭环交互式世界模型,将行为建模与视觉渲染耦合,用于自动驾驶场景生成。该方法先用多智能体强化学习在闭环中微调背景车辆策略,使其与端到端自车规划器的交互行为贴近真实日志,再通过逐 Gaussian 诊断补全未观测视角与表面,并按车辆行驶意图渲染信号灯。在长尾闭环场景中,该方法将交互行为与真实世界的偏差降低 85%,新视角车辆 PSNR 较 SOTA 世界模型提升 2.0dB。

正文 · 原文

View PDF HTML (experimental)

Abstract:World models (WMs) generate driving scenarios beyond recorded logs, enabling the training and evaluation of autonomous driving systems in unseen interactions. However, existing WMs struggle to ensure both behavioral realism and visual fidelity, since they overlook the relationship between vehicle interactions and the ego visual observations. We propose a closed-loop interactive WM that couples behavior modeling and visual rendering. First, imitation-learned background-vehicle (BV) policies are fine-tuned with multi-agent reinforcement learning in the closed loop against an end-to-end ego planner that reacts to the rendered images, so that their interaction behaviors match those in real logs. Second, the viewpoints exposed by these interactions guide a per-Gaussian diagnosis of each 3D Gaussian vehicle asset, which attenuates view-dependent colors on under-observed Gaussians and fills never-observed surfaces with generated Gaussians, while preserving the observed appearance. Third, signal lights are rendered according to the driving intentions of the vehicles, by an editor that learns the light states unobserved for a vehicle through counterfactual fine-tuning with a light-state classifier. On long-tail closed-loop scenarios, our method reduces the divergence from real-world interaction behaviors by 85% and improves novel-view vehicle PSNR by 2.0dB over the state-of-the-art WM. The significance of BV light signaling in WMs is shown, as the rendered signal lights conveying BV driving intentions indirectly impact ego driving behaviors in a quarter of the scenarios.
Comments: 11 pages, 5 figures
Subjects: Robotics (cs.RO); Artificial Intelligence (cs.AI)
Cite as: arXiv:2611.22691 [cs.RO]
  (or arXiv:2611.22691v1 [cs.RO] for this version)
  https://doi.org/10.48550/arXiv.2611.22691

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Yang Chang [view email]
[v1] Tue, 17 Nov 2026 12:39:42 UTC (7,716 KB)

来源:arXiv cs.AI · arxiv.org