arXiv cs.AI· Holger Molin, William Peng, Marco Bagatella, Randall Balestriero·· 3 天前AI 评分31
SpecWM：通过谱对齐实现世界建模
World Modeling through Spectral Alignment
AI 导读
研究者提出 SpecWM，一个 JEPA 框架，通过目标相似度核显式指定观测间的关系，并训练编码器在嵌入几何中复现这些关系。时间谱对齐将 CEM 规划成功率从 LeWorldModel 的 52% 提升至 68%（OGBench Cube）、49% 至 77%（OGBench Scene）、29% 至 51%（CALVIN）。结果表明，显式指定表征几何可引导潜在世界模型保留对控制有用的信息。
正文 · 原文
Abstract:How do we tell a world model what information to preserve? We study this question through the lens of spectral representation learning. We introduce SpecWM, a JEPA framework that explicitly specifies desired relationships between observations through a target similarity kernel and trains the encoder to reproduce those relationships in its embedding geometry. Different kernels provide different supervision, ranging from physical state information to a fully self-supervised temporal kernel requiring only the ordering of observations within trajectories. We characterize the global optima of the objective, establish conditions for linear recoverability and invariance to nuisance information, and connect temporal-kernel similarity to the behavioral successor measure under an idealized setting. Empirically, the choice of kernel strongly controls which physical quantities are recoverable from the representation. More importantly, temporal spectral alignment improves CEM planning success over LeWorldModel from 52% to 68% on OGBench Cube, 49% to 77% on OGBench Scene, and 29% to 51% on CALVIN. These results suggest that explicitly specifying representation geometry provides a simple way to steer what latent world models preserve for control.
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.15429 [cs.LG] |
| (or arXiv:2610.15429v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.15429 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: William Peng [view email]
[v1]
Sun, 11 Oct 2026 08:05:40 UTC (596 KB)
来源：arXiv cs.AI · arxiv.org