arXiv cs.AI· Shiyao Qian, Yuan Ren, Dongfeng Bai·· 2 天前AI 评分22
基于运动与重力感知的前馈 3D 重建框架
Motion- and Gravity-Aware Feedforward 3D Reconstruction
AI 导读
研究者提出一种基于 LUPI 范式的运动与重力感知框架，训练时以残差光流和重力方向作为特权信号引导几何表征学习，推理时仅用标准视觉输入，保持前馈效率。在标准基准上，该方法在含动态物体、结构模糊和复杂相机运动的场景中提升了重建精度与鲁棒性。分析显示最优注入阶段因骨干而异：VGGT 适合中层注入，π3 更适合后期注入。
正文 · 原文
Abstract:Feed-forward transformer models have recently achieved strong performance in multi-view 3D reconstruction, but they still rely primarily on appearance-based attention and often struggle in dynamic scenes with weak geometric cues. In this work, we introduce a motion- and gravity-aware framework under the Learning Using Privileged Information (LUPI) paradigm. During training, residual optical flow and gravity direction are provided as privileged signals to guide geometric representation learning; during inference, the model uses standard visual inputs only.
We integrate these priors into transformer backbones with lightweight architectural changes, preserving feed-forward efficiency. Experiments on standard benchmarks show consistent gains in reconstruction accuracy and robustness, especially in scenes with dynamic objects, ambiguous structure, and challenging camera motion. Our analysis further shows that the optimal injection stage is backbone-dependent: middle-layer integration is generally preferable for VGGT, while late-stage integration is more effective for {\pi}3. These results suggest that LUPI offers a practical way to inject motion and physical priors into feed-forward 3D reconstruction without increasing test-time complexity.
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.22177 [cs.CV] |
| (or arXiv:2610.22177v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2610.22177 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Shiyao Qian [view email]
[v1]
Thu, 15 Oct 2026 16:34:04 UTC (15,458 KB)
来源：arXiv cs.AI · arxiv.org