arXiv cs.CL· Rongguang Ye, Zhan Zhuang, Yichen Wu, Ming Tang, Kede Ma·· 13 小时前AI 评分20
LAPS：为大语言模型持续适应学习锚定提示词空间
Learning an Anchored Prompt Space for Continual Adaptation of Large Language Models
AI 导读
研究者提出 Learning an Anchored Prompt Space（LAPS），通过自蒸馏将历史提示词与更新后的模型对齐，并用可学习的 Bézier 控制提示词构建锚定提示词空间，显式学习任务间提示词关系以促进正向迁移。在 TRACE 基准上，LAPS 于三个 Qwen3 模型规模下均优于基于蒸馏、基于提示词及提示词-参数联合适应的方法，整体任务性能提升且遗忘减少。
正文 · 原文
Abstract:Continually adapting large language models requires acquiring new knowledge while preserving previously learned capabilities. Although jointly adapting model parameters and task-specific soft prompts is promising, it has two key limitations: previously learned prompts may lose effectiveness as the model evolves, and relationships among prompts that could support cross-task transfer are not explicitly learned. We propose Learning an Anchored Prompt Space (LAPS), which preserves historical prompt effectiveness and learns relationships among task-specific soft prompts to facilitate positive transfer. Historical prompts are first aligned with the updated model through self-distillation. LAPS then constructs an anchored prompt space whose vertices correspond to learned task-specific soft prompts and whose intermediate geometry is shaped by learnable Bézier control prompts. Once this space is learned, LAPS identifies the best-performing prompt for each task on its validation set, allowing the selected prompt to draw on knowledge acquired from related tasks. Experiments on the TRACE benchmark across three Qwen3 model scales show that LAPS consistently outperforms existing distillation-based, prompt-based, and joint prompt-parameter adaptation methods, improving overall task performance while reducing forgetting.
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2609.32499 [cs.CL] |
| (or arXiv:2609.32499v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2609.32499 arXiv-issued DOI via DataCite |
Submission history
From: Rongguang Ye [view email]
[v1]
Sat, 26 Sep 2026 11:39:55 UTC (3,278 KB)
[v2]
Tue, 27 Oct 2026 13:08:41 UTC (3,330 KB)
来源：arXiv cs.CL · arxiv.org