arXiv cs.CL· Owen Bianchi, Samer Aslan, Ryunosuke Saito·· 13 小时前AI 评分38
DiaLong：长上下文对话中的记忆召回基准测试
DiaLong: Benchmarking Reminiscence in Long-Context Dialogues
AI 导读
研究团队提出 DiaLong，一个由 100 段人工创建并经 LLM 增强的长对话数据集，配套事实召回任务——对 40 条候选事实判断真假，以评估 LLM 在长对话中主动保持和检索信息的能力。测试最新 LLM 及现有记忆增强方案后发现，模型记忆表现仍有明显不足，需进一步改进。
正文 · 原文
Abstract:How well can transformer-based Large Language Models (LLMs) retain and recall memories in long contexts of dialogue? LLMs are known to forget long-range factual information in their context, yet the field lacks a comprehensive benchmark to quantitatively assess these shortcomings. We present DiaLong, a long-context dialogue dataset and accompanying memory task that standardizes evaluation in this space. It consists of 100 unique conversations created by humans and then augmented with LLMs, extending beyond lengths found in existing datasets. Our selected memory task is fact recall: classify 40 candidate facts about a dialogue as true or false. This task directly probes the model's ability to actively retain and retrieve information, serving as a critical measure of memory within extended conversational contexts. In testing both the latest LLMs and proposed memory enhancements on DiaLong, we demonstrate that more work needs to be done to enable LLMs with better memory performance. By enhancing memory capabilities, LLMs can improve in maintaining coherence, relevance, and human-like interaction over extended conversation. Improving these capabilities will help LLMs fulfill diverse user needs and expectations in various conversational scenarios.
| Comments: | Equal contribution. Work conducted in 2023 at Johns Hopkins University |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.25463 [cs.CL] |
| (or arXiv:2610.25463v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.25463 arXiv-issued DOI via DataCite |
Submission history
From: Samer Aslan [view email]
[v1]
Sun, 18 Oct 2026 18:47:42 UTC (1,058 KB)
[v2]
Tue, 20 Oct 2026 19:55:41 UTC (1,058 KB)
来源：arXiv cs.CL · arxiv.org