arXiv cs.AI· Daphne Cornelisse, Julian Hunt, Zixu Zhang, Wa\"el Doulazmi, Kevin Joseph, Jaime Fern\'andez Fisac, Eugene Vinitsky·· 3 天前AI 评分37
自我对弈加少量人类数据，让驾驶策略学会类人自主性
Human-like autonomy emerges from self-play and a pinch of human data
AI 导读
一项被 CoRL 2026 接收的研究提出，在最小安全到达奖励之上把人类演示当作正则化目标，仅用 30 分钟人类演示数据（比同类模仿学习方法少 2500 倍）即可让自我对弈训练出的驾驶策略与人类轨迹协调一致。该方法在单块消费级 GPU 上 15 小时完成训练，缓解了纯自我对弈策略行为约定与人不兼容的问题。
正文 · 原文
Abstract:Self-play reinforcement learning has recently emerged as a way to train driving policies without any human data. It uses cheap, large-scale simulations to substitute expensive, large-scale human driving demonstrations. A key limitation of this approach is that policies trained through pure self-play can learn effective but alien driving conventions incompatible with people. Previous works attempt to mitigate such behavioral misalignments through extensive reward engineering and domain randomization, which are brittle and labor-intensive. Instead of completely discarding human demonstrations, our method treats them as a regularization objective on top of a minimal safe goal-reaching reward. Like the spice in a good stew, we find that a little human data goes a long way: our method uses only 30 minutes of human demonstrations, 2500x fewer than comparable imitation learning approaches. Resulting policies coordinate with held-out human trajectories and complete training in 15 hours on a single consumer-grade GPU. Videos and full source code are available at this https URL.
| Comments: | 9 pages, accepted to CORL 2026 |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Multiagent Systems (cs.MA) |
| Cite as: | arXiv:2606.19370 [cs.LG] |
| (or arXiv:2606.19370v2 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2606.19370 arXiv-issued DOI via DataCite |
|
| Journal reference: | 10th Conference on Robot Learning (CoRL 2026), Austin TX, USA |
Submission history
From: Daphne Cornelisse [view email]
[v1]
Thu, 11 Jun 2026 19:16:53 UTC (6,632 KB)
[v2]
Mon, 12 Oct 2026 19:48:50 UTC (4,041 KB)
来源：arXiv cs.AI · arxiv.org