arXiv cs.CL· Osamudiamen Edogun, Joshua Obigwe, Irfan Mehmood·· 11 小时前AI 评分38
对话记忆压缩与 RAG 在 LoCoMo 上的配对评测：压缩法领先 6.8 分但依赖答案类型
Compaction, decay and reinforcement in a two-tier conversational memory: a paired evaluation against retrieval-augmented generation
AI 导读
一项配对评测在 LoCoMo 的 1,985 道题上对比"逐轮压缩为原子事实"与检索增强生成（RAG）两种对话记忆方案，压缩法在四个可比类别上得分 71.4% 对 64.6%，领先 6.8 分（p = 5.5e-08），提示词 token 少 34%。
正文 · 原文
Abstract:A conversational agent that remembers has to decide what to keep. Retrieval-augmented generation keeps the transcript and searches it; the alternative is to compact each turn into atomic facts and search those. We compare the two on LoCoMo, with both arms answering all 1,985 questions in one window, against the same stores, with the same generator and judge, and both predictions on one row, which licenses McNemar's exact test on the 512 questions where they disagree.
On the four categories comparable work reports, compaction scores 71.4 per cent against 64.6, a lead of 6.8 points (p = 5.5e-08) at 34 per cent fewer prompt tokens. On all five, including the adversarial set comparable work drops, the lead falls to 2.0 and stops being significant, because the baseline leads adversarial by 14.6.
That loss has a structure. Our lead falls as the reference answer demands more source wording, 9.6 points ahead at one or two reference words and 3.1 behind past eight, with token overlap moving the same way, so the judge is not the cause. Compaction keeps what a fact asserts and discards how it was said, so it wins where the answer is a value and loses where it is a phrasing. We also report four negative results with their power calculations.
| Comments: | 7 figures. Data and code: this https URL |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2611.09807 [cs.CL] |
| (or arXiv:2611.09807v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2611.09807 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Osamudiamen Edogun [view email]
[v1]
Fri, 6 Nov 2026 17:56:37 UTC (79 KB)
来源：arXiv cs.CL · arxiv.org