arXiv cs.AI· Kaixiang Yao, Xu Wang, Miao Pan, Hu Xiyue, Jiaxin Li, Weishi Wang, Daniel Dahlmeier, Jintao Chen, Yongliang Shen, Xuhong Zhang, Jianwei Yin, Yueting Zhuang, Wenqi Zhang·· 3 天前AI 评分27
Spatial-Interactor：通过与环境交互学习空间推理
Spatial-Interactor: Learning Spatial Reasoning through Interaction with the Observable Physical World
AI 导读
研究团队提出 Spatial-Interactor 框架，通过交互让 VLM 建模物理世界状态转移，并构建了 LSI-108K 数据集，覆盖 L1 被动世界状态转移、L2 主动自身状态转移、L3 长程交互轨迹三级课程。
正文 · 原文
Authors:Kaixiang Yao, Xu Wang, Miao Pan, Hu Xiyue, Jiaxin Li, Weishi Wang, Daniel Dahlmeier, Jintao Chen, Yongliang Shen, Xuhong Zhang, Jianwei Yin, Yueting Zhuang, Wenqi Zhang
Abstract:Spatial reasoning is essential for vision-language models (VLMs) to understand and act in the physical world. Reasoning in dynamic environments requires VLMs to perceive local state transitions caused by object motion and viewpoint changes and integrate them over long trajectories to maintain an updated spatial state, yet existing VLMs remain limited in both capabilities. Current spatial training primarily focuses on static questions about object attributes and spatial relations, providing limited direct supervision for state transitions; in contrast, interaction trajectories naturally connect a preceding observation, an action, and a subsequent observation, offering direct supervision for local state transitions, while complete trajectories reveal dependencies among consecutive transitions. We therefore introduce Spatial-Interactor, a framework that trains VLMs to model physical-world state transitions through interaction, organizing this learning process into a three-level curriculum covering L1 passive world-state transitions, L2 active self-state transitions, and L3 long-horizon interaction trajectories. Accordingly, we construct the Learning from Spatial Interaction dataset (LSI-108K) from simulated and real interaction trajectories, with tasks aligned with the objective of each level. Our two-stage training strategy applies Supervised Fine-Tuning (SFT) to L1 and L2 for local transition modeling, and On-Policy Distillation (OPD) then uses privileged self-distillation: a teacher branch given segment-level transition descriptions supervises the student's on-policy CoT, helping the student learn to integrate consecutive transitions over L3 long trajectories. Experiments across multiple VLMs and spatial benchmarks show consistent gains in local transition modeling and long-horizon integration.
| Subjects: | Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2609.23038 [cs.AI] |
| (or arXiv:2609.23038v2 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2609.23038 arXiv-issued DOI via DataCite |
Submission history
From: Kaixiang Yao [view email]
[v1]
Sat, 19 Sep 2026 14:13:59 UTC (37,806 KB)
[v2]
Mon, 12 Oct 2026 09:29:14 UTC (37,807 KB)
来源：arXiv cs.AI · arxiv.org