arXiv cs.CL· Dohoon Kim, Junho Park, Hyunwoong Bae, Seyoon Kim, Taesup Moon·· 12 小时前AI 评分34
Axis-LoRA：为个人记忆检索探测并适配嵌入模型
Axis-LoRA: Probing and Adapting Embedding Models for Personal-Memory Retrieval
AI 导读
研究者提出 Axis-LoRA，在嵌入模型内部放置四个按轴划分的 LoRA 分支，并通过逐查询门控组合，用于区分个人记忆中的时间、地点、人物和事件。在 LongMemEval、ATM-Bench 和 LoCoMo 上评测三种标准嵌入模型，4-benchmark 平均 R@10 提升 +1.28 至 +1.56 pp，且仅训练基座参数的一小部分。代码已开源。
正文 · 原文
Abstract:As LLMs are increasingly asked questions about a user's personal experiences, retrieval must distinguish memories by time, place, person, and event. Current personal-memory pipelines rely on frozen dense embedding models to embed memories for retrieval, but we find that these models do not capture the four axes well. Swapping a memory's date or place barely shifts its embedding, and a trained projection on top of the frozen embedding does not recover the missing signal. The problem lives inside the model, where each axis is encoded by a distinct attention-head pattern. Guided by these findings, we propose Axis-LoRA, a per-axis adapter that places four LoRA branches inside the embedding model and combines them via a per-query gate. Across three standard embedding models evaluated on LongMemEval, ATM-Bench, and LoCoMo, Axis-LoRA improves 4-benchmark mean R@10 by $+1.28$ to $+1.56$~pp while training only a small fraction of the base parameters on a small axis-counterfactual training set. Code is available at this https URL.
| Comments: | EMNLP 2026 Main |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.07820 [cs.CL] |
| (or arXiv:2611.07820v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2611.07820 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Dohoon Kim [view email]
[v1]
Thu, 5 Nov 2026 10:59:23 UTC (1,078 KB)
来源：arXiv cs.CL · arxiv.org