arXiv cs.CL· Vaibhav Prajapati, Pushp Garg·· 2 天前AI 评分22
个性化会破坏信任吗？少样本端侧意图适配下的开放集拒绝校准
Does Personalization Break Trust? Open-Set Rejection Calibration Under Few-Shot On-Device Intent Adaptation
AI 导读
少样本 LoRA 端侧个性化会降低虚拟助手的开放集（OOS）拒绝能力：在五个公开意图分类基准上，OOS 拒绝 AUROC 平均下降 0.055–0.258，而域内准确率保持不变。
正文 · 原文
Abstract:Always-on virtual assistants increasingly personalize on-device to new user-specific commands via few-shot fine-tuning (e.g., LoRA), but this adaptation is evaluated almost exclusively on whether it learns the new commands, not on whether the assistant can still recognize an input that matches none of its intents - the out-of-scope (OOS) rejection problem. We show that naive few-shot LoRA personalization degrades OOS-rejection AUROC (area under the ROC curve) on every one of five public intent-classification benchmarks (mean drop $0.055$--$0.258$, five seeds each), while in-scope accuracy is preserved. A scorer-invariant check shows part of this drop is an artifact of the standard confidence scorer's own miscalibration under adaptation, though a smaller, genuine effect remains under any scorer; we trace it to a measurable shrinkage of the embedding margin around known-intent regions (pooled Spearman rho=0.61$). A deployment-cheap fix is scoring rejection by Mahalanobis distance instead of softmax confidence without extra training/retraining recovers most of the lost quality on 3 of 5 datasets at no accuracy cost, beats two further OOD scoring baselines, and generalizes across shot count, LoRA rank, and a second base encoder. We report honestly where the fix falls short and release all code for reproducibility.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.20652 [cs.CL] |
| (or arXiv:2610.20652v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.20652 arXiv-issued DOI via DataCite |
Submission history
From: Pushp Garg [view email]
[v1]
Mon, 12 Oct 2026 18:54:51 UTC (210 KB)
来源：arXiv cs.CL · arxiv.org