arXiv cs.AI· Zhiyuan Xiang, Peidong Liu, Mingyang Li, Jiawei Li, Wenhow Li, Wenbo Ding, Xiang Deng·· 6 天前AI 评分47
LAPT-Scale：用人类视频潜在动作预训练扩展具身策略
LAPT-Scale: Scaling Embodied Policies with Latent Action Pre-Training on Human Videos
AI 导读
LAPT-Scale 提出可扩展的潜在动作预训练范式，其中 LAPT-Scale-LAM 通过解释人类视频帧间视觉变化学习连续潜在动作，为 VLA 策略预训练提供隐式动作监督。在 23.3K 小时第一人称视频上训练后，标注人类视频从 1K 扩展到 5K 小时，真实机器人成功率从 63.3% 提升至 85.0%，并泛化到未见视觉条件。该工作已被 CoRL 2026 接收。
正文 · 原文
Abstract:Scaling generalist robotic policies is limited by the high cost of collecting embodied interaction data. Large-scale egocentric human videos offer a rich alternative, but they lack robot action labels, and existing action annotation methods are often noisy under real-world visual complexity and diverse manipulation dynamics. We introduce LAPT-Scale, a scalable latent action pre-training paradigm with two components: LAPT-Scale-LAM learns continuous latent actions by explaining frame-to-frame visual transitions in human videos and uses them as human-derived supervision, while LAPT-Scale-Policy trains a VLA policy through human-video latent-action pre-training, target-robot embodiment alignment, and task-specific post-training. Unlike prior vector-quantized LAMs, LAPT-Scale-LAM uses a continuous latent space to preserve fine-grained motion and interaction dynamics. We further combine perceptual, semantic, and optical-flow alignment signals to improve transition-level supervision beyond pixel reconstruction. Trained on 23.3K hours of egocentric video, LAPT-Scale-LAM provides implicit action supervision for policy pre-training, yielding consistent scaling gains and improving real-robot success from 63.3% to 85.0% as annotated human video scales from 1K to 5K hours. The learned policy also generalizes to unseen visual conditions.
| Comments: | Accepted at CoRL 2026; 24 pages including appendix |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Robotics (cs.RO) |
| Cite as: | arXiv:2610.13732 [cs.CV] |
| (or arXiv:2610.13732v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2610.13732 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: ZhiYuan Xiang [view email]
[v1]
Fri, 9 Oct 2026 15:29:20 UTC (6,528 KB)
来源：arXiv cs.AI · arxiv.org