arXiv cs.AI· Binay Kumar Singh, Niels Da Vitoria Lobo·· 10 小时前AI 评分20
自动驾驶世界模型的动作条件语义预测是否真正依赖动作?WACV 2027 WAM-AD Workshop 论文探究
Does the Action Matter? Probing Action-Conditioned Semantic Forecasting as a World Model for Autonomous Driving
AI 导读
研究对比了带动作条件与不带动作输入的语义预测模型,用测试时动作干预衡量动作依赖。实验显示正确动作最多提升中期 mIoU 2.5 点(Cityscapes)和 0.6 点(BDD100K),错误动作则分别降低最多 23.2 和 7.0 点,低于无动作输入模型。真实数据上特征级条件比输入级条件对动作的依赖高 1.6 倍,说明精度提升只能证明模型用到了动作,不能说明依赖强度。
正文 · 原文
Abstract:World models are increasingly used in autonomous driving to predict how a scene evolves under the ego vehicle's actions, supporting counterfactual reasoning and model-based planning. However, existing evaluations focus on prediction fidelity and planning performance, and rarely verify that the predictions depend on the conditioning action. In this paper, we study action dependence in action-conditioned semantic forecasting, which predicts future semantic segmentation from past segmentations and the ego action. We compare a forecaster with feature-wise and input-level action conditioning against an identical model without action input, and measure action dependence with test-time action interventions scored by standard segmentation metrics. The evaluation is validated in a synthetic environment with known action-dependent dynamics and applied to Cityscapes and BDD100K, both within each dataset and across the two vehicle platforms. Experiments show that the executed action improves mid-term mIoU by at most 2.5 points on Cityscapes and 0.6 points on BDD100K, whereas a wrong action reduces it by up to 23.2 and 7.0 points, below the model without action input. On real data, feature-wise conditioning depends 1.6 times more on the action than input-level conditioning at similar accuracy, whereas both depend on it equally in the synthetic environment, where the history does not determine the action. These results show that an accuracy gain indicates that a driving world model uses its action, but not how strongly its forecasts depend on it. Code is available at this https URL.
| Comments: | 8 pages, 3 figures, 6 tables. Accepted at the WAM-AD Workshop, WACV 2027. Code: this https URL |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Robotics (cs.RO) |
| Cite as: | arXiv:2611.24840 [cs.CV] |
| (or arXiv:2611.24840v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2611.24840 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Binay Singh [view email]
[v1]
Wed, 18 Nov 2026 18:06:20 UTC (116 KB)
来源:arXiv cs.AI · arxiv.org