Amazon 论文提出 FEB-30 基准：仅靠 QA 分数无法评估智能体记忆系统
Amazon 论文提出 FEB-30 基准，用 30 条脚本化事实轨迹生成 147 篇文档，在每步已知正确记忆状态的前提下，直接对比存储事实与 QA 分数。在类 Mem0 和 Hindsight 的记忆系统上，两种评估平均相差 16.8 分，21% 的事实轨迹通过/失败判定不一致；系统选对更新、强化等记忆操作的比例仅 20.1%，而 QA 评估无法发现这一点。
Rohan Paul· @rohanpaul_ai · X·2026-10-19 14:02· 1 小时前AI 评分50
AI 导读
Amazon 论文提出 FEB-30 基准，用 30 条脚本化事实轨迹生成 147 篇文档，在每步已知正确记忆状态的前提下直接对比存储事实与 QA 分数。在类 Mem0 和 Hindsight 的记忆系统上，两种评估平均相差 16.8 分，21% 的事实轨迹通过/失败判定不一致，系统选对更新、强化等记忆操作的比例仅 20.1%，而 QA 评估无法发现这一点。
正文 · 原文
New Amazon paper shows that teams shipping agent memory should check what actually got stored, since QA scores alone did not reveal which pipeline stage was failing.
Agent memory benchmarks grade the agent's answers, which mixes memory with retrieval and answer writing. A strong LLM can hide a broken memory store, and a retrieval miss can make a correct one look bad.
They built FEB-30, 30 scripted fact histories turned into 147 documents, so the correct memory state is known after every document. On a memory system built like Mem0 and Hindsight, they compared the stored facts with that truth and with QA scores.
The 2 scores differed by 16.8 points on average, and pass/fail verdicts disagreed on 21% of fact histories. The system picked the right memory action, like update or reinforce, only 20.1% of the time, which QA never showed.
If you build agent memory, test the stored facts directly after each update, alongside end-to-end QA.
来源：Rohan Paul · x.com
来源：RadarAI · aihot.news