跳到正文
arXiv cs.CL· Son Duong, Truong X. Tran, Hai-Anh Tran·· 10 小时前AI 评分22

ICARE:用增量类别感知表示与蒸馏学习做跨域文本分类

ICARE: cross-domain text classification with incremental class-aware representation and distillation learning

AI 导读

研究提出 ICARE 框架,用于跨域文本分类中的持续多任务学习,包含基于表示学习的实例选择模块和特征级与 logit 级知识蒸馏方法,以缓解灾难性遗忘。在基准文本分类数据集上,ICARE 显著提升目标准确率并降低“遗忘指标”,源代码已公开。

正文 · 原文

View PDF HTML (experimental)

Abstract:The rise of powerful, fine-tuned large language models (LLMs) necessitates robust natural language inference (NLI) systems. These NLI systems ideally train across various tasks, encompassing diverse concepts like sentiment analysis, news classification, and coreference resolution. However, this continual, multi-tasking approach can lead to catastrophic forgetting, where previously learned tasks are lost, especially in domains lacking specific labels. Rehearsing past examples with labels can further exacerbate this issue by causing overfitting, hindering the system's ability to classify new categories. To tackle these challenges, this research proposes ICARE, a novel framework with two key components: an instance selection module and a knowledge distillation method. The former is based on representation learning, which meticulously selects training examples to present to the NLI system. The latter, knowledge distillation with feature and logit-level transfer, facilitates knowledge transfer between tasks. This leverages pruning techniques and optimized objective functions to transfer both essential feature-level and decision-making (logit-level) knowledge, mitigating catastrophic forgetting while adapting to new patterns. Extensive evaluations on benchmark text classification datasets have demonstrated ICARE's effectiveness. The framework achieves a significant improvement in target accuracy and reduction in the ``forgetting metric''. These results solidify ICARE's ability to preserve and leverage learned knowledge across diverse tasks, ultimately enhancing the adaptability and performance of NLI systems. The source code is available at this https URL
Subjects: Computation and Language (cs.CL); Machine Learning (cs.LG)
Cite as: arXiv:2611.24959 [cs.CL]
  (or arXiv:2611.24959v1 [cs.CL] for this version)
  https://doi.org/10.48550/arXiv.2611.24959

arXiv-issued DOI via DataCite (pending registration)

Journal reference: International Journal of Machine Learning and Cybernetics, vol. 16, pp. 5277-5294, 2025
Related DOI: https://doi.org/10.1007/s13042-025-02572-6

DOI(s) linking to related resources

Submission history

From: Truong Tran [view email]
[v1] Wed, 11 Nov 2026 13:42:02 UTC (488 KB)

来源:arXiv cs.CL · arxiv.org