arXiv cs.CL· Yanhao Jia, Yi Zhu, Yiran Zhong, Weigao Sun, Feida Zhu, Pengkun Jiao, Sihan Yang, Ming Ma, Steven Hoi, Erik Cambria·· 12 小时前AI 评分38
4DMemory：用 Harness 增强智能体评测长时程时空记忆
4DMemory: Benchmarking Long-Horizon Spatiotemporal Memory with Harness-Augmented Agents
AI 导读
研究团队提出 LongVideo 数据集与 4DMemoryBench，含约 900 小时视频和 5000 多道人工审核问题，覆盖视觉、文本与时空推理。配套的 4DMemory Harness 整合 4 项技能、22+ 专用工具模型和模型专属 RAG 系统，使 13 个受测模型全部提升超过 50%。这些模型在 4DMemoryBench 上相比此前基准性能下降逾 10%。
正文 · 原文
Abstract:Humans efficiently process multimodal streams spanning hours to years, building persistent, fine-grained spatiotemporal memories that support reasoning about and predicting long-horizon events. In contrast, existing benchmarks focus on short- and medium-context tasks, and most foundation models struggle to preserve fine-grained information over long horizons. To evaluate and improve long-horizon spatiotemporal reasoning, we introduce the LongVideo dataset and the 4DMemoryBench. LongVideo comprises about 900 hours of video and over 5,000 manually reviewed questions covering visual, textual, and spatiotemporal reasoning across diverse context lengths. We further develop the 4DMemory Harness, which integrates four complementary skills, 22+ specialized tool models, and a model-specific retrieval-augmented generation system. Evaluating 13 leading models, we find that standalone foundation models struggle with long-horizon tasks, while the 4DMemory Harness improves all of them by over 50\%. Compared to previous benchmark, the same models exhibit a performance drop of more than 10\% on 4DMemoryBench. We hope 4DMemory will facilitate the development of spatiotemporal agents capable of addressing complex, long-horizon tasks.
| Comments: | 22 pages, 5 figures |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Multimedia (cs.MM) |
| Cite as: | arXiv:2610.32332 [cs.CV] |
| (or arXiv:2610.32332v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2610.32332 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Yanhao Jia [view email]
[v1]
Fri, 23 Oct 2026 08:28:29 UTC (3,283 KB)
来源：arXiv cs.CL · arxiv.org