arXiv cs.CL· Md Tahmid Rahman Laskar, Xue-Yong Fu, Shashi Bhushan TN·· 12 小时前AI 评分34
一个微调模型能否包揽所有任务？客服 LLM 的微调策略选择研究
Can One Adapted Model Do It All? Fine-Tuning Strategy Selection for Customer Support LLMs
AI 导读
研究用 Qwen3、Qwen3.5、Gemma-3、Llama-3.1、Mistral 五个家族 0.6B 到 32B 的 13 个模型、8 个客服数据集（约 74.5k 训练样本、8.7k 评测样本）训练超 200 个 checkpoint，发现多任务全量微调在各模型规模下都是最强的默认策略。
正文 · 原文
Abstract:Production customer-support systems often require LLMs to support multiple skills, such as intent classification, question answering, summarization, or tool-use decisions. A central deployment question is whether these skills should be handled by separate task-specialist models or by a single model trained through multi-task training, sequential updates, or model merging. We study this question using thirteen models spanning five families (Qwen3, Qwen3.5, Gemma-3, Llama-3.1, and Mistral) from 0.6B to 32B parameters across eight customer-support datasets, spanning four public and four proprietary datasets with approximately 74.5k training and 8.7k evaluation samples. Under a fixed training protocol, we train more than 200 checkpoints. Our experiments reveal that multi-task full fine-tuning is the strongest operational default at every model size we test. Specialist models are strong on their target tasks but often degrade sharply off-task, making reliable routing important. Sequential Low-Rank Adaptation (LoRA) preserves earlier skills better than sequential full fine-tuning, while merging a specialist with its base model improves off-task robustness with limited same-task loss for larger models. We conclude with practical guidelines for selecting fine-tuning strategies in real-world settings.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2609.27262 [cs.CL] |
| (or arXiv:2609.27262v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2609.27262 arXiv-issued DOI via DataCite |
Submission history
From: Md Tahmid Rahman Laskar [view email]
[v1]
Wed, 23 Sep 2026 02:46:50 UTC (89 KB)
[v2]
Wed, 4 Nov 2026 02:13:40 UTC (89 KB)
来源：arXiv cs.CL · arxiv.org