arXiv cs.CL· Hong Ting Tsang, Boyi Zhang, Haoyu Huang, Jiaxin Bai, Yufei Li, Yisen Gao, Zhongwei Xie, Hongyu Luo, Jeff Z. Pan, Yangqiu Song·· 13 小时前AI 评分38
MOSAIC：面向对话智能体长期记忆与跨模态推理的全模态基准
MOSAIC: An Omni-Modal Benchmark for Long-Term Memory and Cross-Modal Reasoning in Conversational Agents
AI 导读
研究者提出 MOSAIC，一个覆盖文本、图像、视频、音频的全模态长期记忆基准，包含 923 道问题，基于 11-24 轮会话的人设化历史。该基准构建空间、时间与韵律信号，并配套诊断协议，在固定问题、阅读器、提示词与预算的前提下分别干预表示、检索与交付环节。匹配分解显示表示损失与模态相关，即便给定金标证据阅读残差仍占主导，且模型在拒答控制上表现薄弱。
正文 · 原文
Abstract:Omni-modal assistants must remember not only what users say, but also what they show and convey. Existing long-term memory benchmarks are mostly text-and-image or report a single end-to-end score, so they cannot locate whether evidence is lost when it is represented, retrieved, delivered, or read. We introduce MOSAIC, a benchmark of 923 questions over persona-grounded histories of 11-24 sessions spanning text, image, video, and audio. MOSAIC constructs targeted spatial, temporal, and prosodic signals rather than decorative media, with the strongest causal validation for spatial information and selector-level validation for annotated prosody, and pairs them with a matched diagnostic protocol that freezes the question, reader, prompt, and budget while intervening on representation, retrieval, and delivery. Matched decomposition shows that representation loss is modality-dependent, residual reading dominates even under gold evidence, and abstention remains weak on these controls: strong answerable scores coexist with failures to refuse. MOSAIC thus turns omni-modal memory evaluation from a leaderboard into a diagnosis of write, retrieve, deliver, and read failures.
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.29249 [cs.CL] |
| (or arXiv:2610.29249v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.29249 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Hong Ting Tsang [view email]
[v1]
Wed, 21 Oct 2026 05:55:20 UTC (4,850 KB)
来源：arXiv cs.CL · arxiv.org