arXiv cs.AI· Yeseul Cho, Hanseul Cho, Chulhee Yun·· 13 小时前AI 评分26
Error-Subspace Calibration（ESCal）：面向多任务模型合并的误差子空间校准
Error-Subspace Calibration for Multi-Task Model Merging
AI 导读
研究者提出合并后校准框架 ESCal，从专家模型与合并模型的权重差异中识别低维误差子空间，在每个中间块边界用桥接模块将合并表示投影到下一块输入投影的误差子空间，再映射为当前块输出投影误差子空间中的加性修正。子空间基保持固定，校准只学习这些线性映射，参数效率高。在视觉与自然语言理解基准上，ESCal 用每任务少量校准集即可提升多种架构与合并方法的合并模型性能。
正文 · 原文
Abstract:Model merging combines task-specific expert models into a single model without joint training on their original datasets, but often incurs a performance gap relative to the individual experts. Post-merging calibration can reduce this gap through task-specific representation alignment, but the alignment objective alone does not specify which feature directions to correct. We propose Error-Subspace Calibration (ESCal), a post-merging calibration framework that identifies low-dimensional error subspaces from weight differences between expert and merged models. At each intermediate block boundary, a bridge module projects the merged representation onto the next block's input-projection error subspace and maps these coordinates to an additive correction in the current block's output-projection error subspace. The subspace bases remain fixed, so calibration learns only these linear maps, yielding a parameter-efficient mechanism for feature correction. We find that ESCal brings the merged model closer to the experts in both representations and predictions, indicating more faithful recovery of task-specific behavior. Experiments on vision and natural language understanding benchmarks show that ESCal improves merged-model performance across diverse architectures and merging methods using a small calibration set for each task.
| Comments: | 25 pages |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.33483 [cs.LG] |
| (or arXiv:2610.33483v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.33483 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Yeseul Cho [view email]
[v1]
Sat, 24 Oct 2026 06:17:15 UTC (447 KB)
来源：arXiv cs.AI · arxiv.org