跳到正文
arXiv cs.AI· Xianyao Li, Ruitong Tian, Rui Min, Fang Xu, Jing Du·· 4 小时前AI 评分31

可执行感知记忆 EPM:让世界模型在想象中保留所见所触

Perception into Imagination: Executable Memory for World Models

AI 导读

研究者提出可执行感知记忆(EPM)世界模型框架,在想象过程中将显式感知状态与神经潜在动力学同步推进,其演化读出用于奖励预测、价值估计和策略学习。在 Memory Maze 13×13 上,EPM 于 1M 次交互达到 30.37 回报,而相同感知前端下仅用该读出指导策略与价值时为 23.37,提升 7.00 [3.80, 10.21]。

正文 · 原文

View PDF HTML (experimental)

Abstract:Physical agents must use what they previously saw or touched after visibility or contact is lost. Their actions change the implications of these remembered facts. We introduce executable perceptual memory (EPM), a world-model framework that advances explicit perceptual state alongside neural latent dynamics during imagination. Its evolving read-out informs reward prediction, value estimation and policy learning. We align reward inputs with the transitions that produced them and supervise the reward head on open-loop rollouts of recorded actions. On Memory Maze $13\times13$, EPM reaches 30.37 return at 1M interactions, compared with 23.37 when the same evolving read-out informs only policy and value on a common perceptual front end: a gain of 7.00 [3.80, 10.21]. The complete method also improves control on $9\times9$ and in MiniWorld. Probes of remembered, out-of-view arrivals show stronger reward anticipation in both visual implementations. Concentration, tactile recovery and real tactile frames demonstrate complementary roles of remembered facts in imagined-policy training, motion-dependent decisions and sensory-content retention. EPM connects persistent perception to the predicted consequences and learning signals of a world model for Physical AI.
Comments: 18 pages, 6 figures, 15 tables
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV); Robotics (cs.RO)
Cite as: arXiv:2611.22294 [cs.LG]
  (or arXiv:2611.22294v1 [cs.LG] for this version)
  https://doi.org/10.48550/arXiv.2611.22294

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Xianyao Li [view email]
[v1] Tue, 17 Nov 2026 06:52:02 UTC (390 KB)

来源:arXiv cs.AI · arxiv.org