arXiv cs.AI· Steve Hanneke, Samory Kpotufe·· 12 小时前AI 评分20
迁移学习中的源数据与目标数据聚合价值：监督场景下的弱迁移与强迁移二分
Help Me Help You: The Aggregate Value of Source and Target Data in Transfer Learning
AI 导读
研究者在监督式迁移学习（目标域有标注数据）中识别出两种速率区间：弱迁移区间的最优速率仅由源数据或目标数据之一的信息决定，强迁移区间则可利用两者互补信息实现更快的目标域收敛。作者提出“弱/强迁移模数”细化相关性概念，刻画哪些分布对 P 与 Q 属于哪种区间，并据此重新分类现有算法，给出无需先验分布知识、在各区间均近最优的通用方法。
正文 · 原文
Abstract:Transfer learning, in regression or classification, concerns problems where one aims to use imperfect data from a source distribution P to improve prediction w.r.t. a target distribution Q. While there is now much theoretical understanding of the unsupervised setting of this problem (where no labeled target data is available), the supervised setting with labeled target data remains under-studied.
We identify and characterize an interesting dichotomy in the rates achievable in the supervised setting: there is a weak transfer regime where optimal rates are determined by information in either the source or target dataset but not both, and a strong transfer regime where much faster target rates are achievable, accounting for complementary information in source and target data. Furthermore, while previous theoretical tools for the unsupervised setting can be extended to capture the weak transfer regime, we argue that they cannot capture the strong regime due to limitations inherent in the various notions of relatedness they rely on. Instead, we provide an appropriate refinement of relatedness notions---termed weak and strong moduli of transfer---that reveal the two regimes and lead to a nontrivial characterization of which pairs of distributions P and Q adhere to which regime.
Finally, the above dichotomy reveals a new categorization of existing algorithmic approaches in terms of their optimality or lack-thereof across supervised regimes, and we provide a generic approach that remains nearly optimal in all regimes without prior distributional knowledge.
| Comments: | This is a revision of a work previously posted to arXiv in 2024 by the same authors under the same arXiv number with a different title. The current version is revised to better highlight the key contributions. No AI was used in the production of this work |
| Subjects: | Machine Learning (stat.ML); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Statistics Theory (math.ST) |
| Cite as: | arXiv:2408.16189 [stat.ML] |
| (or arXiv:2408.16189v4 [stat.ML] for this version) | |
| https://doi.org/10.48550/arXiv.2408.16189 arXiv-issued DOI via DataCite |
Submission history
From: Steve Hanneke [view email]
[v1]
Thu, 29 Aug 2024 01:02:40 UTC (32 KB)
[v2]
Mon, 28 Apr 2025 03:15:44 UTC (242 KB)
[v3]
Fri, 25 Sep 2026 23:49:00 UTC (252 KB)
[v4]
Tue, 20 Oct 2026 01:20:19 UTC (252 KB)
来源：arXiv cs.AI · arxiv.org