清华联合新浪微博提出 BEMULA 基准，评测 LLM Agent 记忆遗忘能力
清华 BEMULA 基准揭示 LLM Agent 几乎无法按要求彻底遗忘记忆
清华大学联合新浪微博提出 BEMULA 基准，用于评测 LLM Agent 的记忆遗忘能力，包含 50 段多轮对话、135 个遗忘请求和 1742 个测试问题，覆盖 6 种记忆系统和 3 个骨干模型。
Rohan Paul· @rohanpaul_ai · X·2026-11-04 18:37· 1 小时前AI 评分64
AI 导读
清华大学联合新浪微博提出 BEMULA 基准，评测 LLM Agent 的记忆遗忘能力，包含 50 段多轮对话、135 个遗忘请求和 1742 个测试问题，覆盖 6 种记忆系统和 3 个骨干模型。
正文 · 原文
New Tsinghua paper finds that when people ask AI agents with long-term memory to forget a personal detail, the assistants almost never remove it cleanly, either keeping it or deleting too much.
Agents with long-term memory store personal details, and users increasingly ask them to delete things like a home address. Existing memory benchmarks test only storing and recalling facts.
They built BEMULA, 50 long synthetic chats with 135 forget requests, each paired with a copy that has no request. Probes check whether the fact is gone, other memories survive, and attacks can still extract it.
No tested setup passed more than 1.5% of forget requests end to end, across 6 memory systems and 3 backbone models. After a forget request, systems blocked at most 35.7% of simple extraction prompts like role-play or yes-or-no guesses.
If your agent remembers users, build and test a real delete path, since a confirmation message does not mean the data is gone.
来源：Rohan Paul · x.com
来源：RadarAI · aihot.news