arXiv cs.AI· Yixuan Xie, Xueqiang Xu, Xinyi Fan, Xinxi Lyu, Tao Feng, Xing Fan, Chenlei Guo, Jie Hao, Shaowen Wang, Jiashuo Sun, Jiawei Han·· 2 天前AI 评分37
MemComposer：用强化学习为 LLM 智能体组装记忆
MemComposer: Learning to Assemble Memories for LLM Agents via Reinforcement Learning
AI 导读
MemComposer 通过联合考察当前查询与检索候选，决定纳入哪些经验、选多少条以及如何排序，并用基于 GRPO 的 Experience Policy Optimization（ExPO）以完整智能体交互的任务成功反馈训练组装策略。
正文 · 原文
Authors:Yixuan Xie, Xueqiang Xu, Xinyi Fan, Xinxi Lyu, Tao Feng, Xing Fan, Chenlei Guo, Jie Hao, Shaowen Wang, Jiashuo Sun, Jiawei Han
Abstract:Agent memory enables large language model (LLM) agents to reuse past experiences without updating their parameters. However, semantic similarity alone does not guarantee memory usefulness: downstream performance depends on how memories are selected and presented to the agent. We propose MemComposer, a memory composer that jointly examines the current query and retrieved candidates to decide which experiences to include, how many to select, and how to order them. We train the composer using Experience Policy Optimization (ExPO), a GRPO-based framework that evaluates assembled memory contexts through complete agent interactions and uses task-success feedback to optimize the assembly policy. Across three $\tau^2$-bench domains with two frozen executors, MemComposer outperforms the strongest baseline for each executor while reducing average memory-token usage by 16.9% and 4.9%, respectively. The learned policy also transfers to unseen domains, executors, and memory banks, and benefits from memories generated by stronger models without further training. These results demonstrate the value of learning query-specific memory assembly using downstream task feedback.
| Comments: | 30 pages, 8 figures, 14 tables |
| Subjects: | Artificial Intelligence (cs.AI); Computation and Language (cs.CL) |
| ACM classes: | I.2.7; I.2.6 |
| Cite as: | arXiv:2610.21091 [cs.AI] |
| (or arXiv:2610.21091v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.21091 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Xueqiang Xu [view email]
[v1]
Thu, 15 Oct 2026 01:47:06 UTC (468 KB)
来源：arXiv cs.AI · arxiv.org