arXiv cs.CL· Weiyang Jin, Jiwen Yu, Sihan Xu, Yuwei Niu, Xihui Liu·· 12 小时前AI 评分31
Execution Before Memory：诊断视频世界模型中的记忆失效问题
Execution Before Memory: Diagnosing Memory Failures in Video World Models
AI 导读
研究提出 MIST 评测，将“场景设定”与“指令变更执行”分离，发现视频世界模型的瓶颈在于模型能设定场景却未执行指令变更，而非记忆遗忘。基于此提出 M3-Agent（Make the Missing Move），通过递归改写输入指令细节让模型完成所要求的变更，MIST reveal 分数提升超过三倍，WRBench 的 T 分数提升 26.1%，MBench-T 的 U 分数提升 52.7%。
正文 · 原文
Abstract:A memory test for a video world model should say whether the scene was set, whether the asked change was carried out, and whether the last frame shows the result. Current tests fold these into one number, so a low score is easy to read as forgetting. When the change is not carried out, the hidden state transition cannot appear, so the test is not really a test of memory. We introduce MIST, an evaluation that separates the change from the memory it should leave behind. The model first sets the scene while the change stays hidden, and only later is asked to show it, with leakage before that point kept under control. This separation locates the bottleneck: the model sets the scene but does not carry out the instructed change. The same gap appears in general memory evaluations. On that basis we propose M3-Agent (Make the Missing Move), which recursively revises the details of the input instruction so that the model successfully carries out the requested instruction. Relative to the baseline, it more than triples the MIST reveal score. On the benchmarks, WRBench's score T rises by 26.1% and MBench-T's score U by 52.7%. The method meets the problem from the execution side: it shows what the frozen model can do once the move is stated, and it sets a stricter baseline.
| Comments: | 27 pages, 7 figures. Project page: this https URL Code: this https URL |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL) |
| ACM classes: | I.2.10; I.4.8 |
| Cite as: | arXiv:2610.23542 [cs.CV] |
| (or arXiv:2610.23542v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2610.23542 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Weiyang Jin [view email]
[v1]
Fri, 16 Oct 2026 14:37:13 UTC (4,526 KB)
来源：arXiv cs.CL · arxiv.org