arXiv cs.AI· Madelon Bernardy, Mohammad Mahdi Derakhshani, David M Knigge, Riccardo Valperga, Artem Moskalev·· 11 小时前AI 评分31
让策略编码器对齐形状教师：纯图像策略如何习得 3D 形状信息
Aligning Policy Encoders with a Shape Teacher
AI 导读
研究提出 shape-teacher alignment，在训练阶段用预训练 3D 形状表示 Point-MAE 的物体特征作为图像 patch 的对齐目标，把几何结构迁移给图像策略编码器，部署时无需形状教师或深度输入。在 LIBERO 上，该方法使图像表示更围绕物体形状与部件结构化，预训练策略在 Object 和 Spatial 套件上的收敛速度分别提升 4.7x 和 5.6x，成功率同步提高。
正文 · 原文
Abstract:3D-aware representations can improve robotic manipulation by providing geometric information that is robust to changes in viewpoint, object position, and occlusion. However, using 3D inputs at deployment requires additional sensing and processing, making image-based policies more practical. We ask whether an image-based policy can acquire 3D shape information during training while retaining a purely image-based interface at deployment. We propose shape-teacher alignment, a training-time mechanism that uses a pretrained 3D shape representation as an alignment target for an image-based policy encoder. Specifically, we use Point-MAE features from individual objects as targets for corresponding image patches, transferring geometric structure to the policy without changing its deployment inputs or architecture. On LIBERO, shape-teacher alignment produces image representations that are more structured around object shape and corresponding parts, while substantially accelerating policy convergence and improving success rates. For pretrained policies, convergence is accelerated by 4.7x and 5.6x on the Object and Spatial suites, respectively, without requiring the shape teacher or depth input at deployment.
| Subjects: | Robotics (cs.RO); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV) |
| Cite as: | arXiv:2611.08275 [cs.RO] |
| (or arXiv:2611.08275v1 [cs.RO] for this version) | |
| https://doi.org/10.48550/arXiv.2611.08275 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Angela Bernardy [view email]
[v1]
Thu, 5 Nov 2026 16:44:24 UTC (2,329 KB)
来源：arXiv cs.AI · arxiv.org