arXiv cs.AI· Kangcong Li, Peng Ye, Lin Zhang, Chao Wang, Huafeng Qin, Jiayuan Fan, Tao Chen·· 9 天前AI 评分23
FreshMem：面向流式视频理解的脑启发频率-空间混合记忆网络
FreshMem: Brain-Inspired Frequency-Space Hybrid Memory for Streaming Video Understanding
AI 导读
FreshMem 是一种受大脑对数感知与记忆巩固机制启发的频率-空间混合记忆网络，用于将 MLLM 从离线扩展到在线流式视频理解。它通过多尺度频率记忆（MFM）和空间缩略图记忆（STM）两个模块兼顾短期保真与长期连贯，在 StreamingBench、OV-Bench、OVO-Bench 上分别将 Qwen2-VL 基线提升 5.20%、4.52% 和 2.34%。
正文 · 原文
Abstract:Transitioning Multimodal Large Language Models (MLLMs) from offline to online streaming video understanding is essential for continuous perception. However, existing methods lack flexible adaptivity, leading to irreversible detail loss and context fragmentation. To resolve this, we propose FreshMem, a Frequency-Space Hybrid Memory network inspired by the brain's logarithmic perception and memory consolidation. FreshMem reconciles short-term fidelity with long-term coherence through two synergistic modules: Multi-scale Frequency Memory (MFM), which projects overflowing frames into representative frequency coefficients, complemented by residual details to reconstruct a global historical "gist"; and Space Thumbnail Memory (STM), which discretizes the continuous stream into episodic clusters by employing an adaptive compression strategy to distill them into high-density space thumbnails. Extensive experiments show that FreshMem significantly boosts the Qwen2-VL baseline, yielding gains of 5.20%, 4.52%, and 2.34% on StreamingBench, OV-Bench, and OVO-Bench, respectively. As a training-free solution, FreshMem outperforms several fully fine-tuned methods, offering a highly efficient paradigm for long-horizon streaming video understanding.
| Comments: | Accepted by NeurIPS 2026 |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2602.01683 [cs.CV] |
| (or arXiv:2602.01683v2 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2602.01683 arXiv-issued DOI via DataCite |
Submission history
From: Kangcong Li [view email]
[v1]
Mon, 2 Feb 2026 05:52:11 UTC (24,890 KB)
[v2]
Thu, 8 Oct 2026 05:09:57 UTC (24,957 KB)
来源：arXiv cs.AI · arxiv.org