arXiv cs.AI· Qixin Sun, WenLong Gao, Xiansheng Chen, Zhongyuan Wang, Si Liu, Pengwei Wang·· 2 天前AI 评分38
LTM：面向视觉-语言-动作模型的潜在转移上下文记忆
LTM: Latent Transition Based In-Context Memory for Vision-Language-Action Models
AI 导读
研究者提出 Latent Transition Memory（LTM），将历史存为相邻观测间的紧凑潜在转移，并在 π₀.₅ 上实例化为 LTM-π，无需记忆专用语义标注或外部推理模型调用。
正文 · 原文
Abstract:Robotic manipulation often requires remembering interactions that are no longer visible. We introduce Latent Transition Memory (LTM), which stores history as compact latent transitions between consecutive observations. Instantiated on $\pi_{0.5}$ as LTM-$\pi$, our method uses a frozen latent action encoder, a lightweight adapter, and causal KV caching for incremental inference without memory-specific semantic annotations or external reasoning-model calls. With DreamDojo transitions, LTM-$\pi$ achieves 58.54% success on RoboMME and 80.5% on LIBERO-Mem, performing best among baselines without external reasoning models or memory-specific annotations. Ablations support the contribution of transition content beyond temporal cues and the efficiency of delayed memory admission. Scaling RoboMME demonstrations tenfold improves success from 57.67% to 87.50%, while a Cup Swap diagnostic highlights the role of process supervision in temporal composition. Under a 200-token memory budget, success remains 84.00%, versus 87.50% without compression. Finally, real-robot counting success improves from 11.25% to 82.50%. Together, these results demonstrate the effectiveness, scalability, and efficiency of latent transition memory, highlighting its capacity for complex temporal information, bounded-context operation, and real-world robotic control.
| Comments: | Dataset & Code: this https URL |
| Subjects: | Robotics (cs.RO); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.21362 [cs.RO] |
| (or arXiv:2610.21362v1 [cs.RO] for this version) | |
| https://doi.org/10.48550/arXiv.2610.21362 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Qixin Sun [view email]
[v1]
Thu, 15 Oct 2026 06:41:20 UTC (2,957 KB)
来源：arXiv cs.AI · arxiv.org