arXiv cs.CL· Guodong Du, Wanyu Lin·· 12 小时前AI 评分34
DiDi-Merging：动态模型合并的瘦身方案
Dynamic Model Merging Made Slim
AI 导读
DiDi-Merging 通过可微秩选择为共享参数与专家参数联合分配低秩容量，并进行无数据参数空间精修。在八任务 ViT-B/32 基准上，小型配置以单骨干 1.24 倍存储达到 89.3% 平均准确率（98.7% 归一化保留率），中型配置以 1.4 倍存储达到 89.8%。移除精修会使中型配置下降 4.3 个百分点，静态合并加均匀恢复的对照组比小型配置低 4.7-8.9 个百分点。
正文 · 原文
Abstract:Model merging consolidates independently fine-tuned checkpoints without joint training or access to task data. Dynamic methods retain task-specific capacity, but must divide a limited storage budget between shared and expert parameters. We propose DiDi-Merging, which jointly allocates low-rank capacity to these components through differentiable rank selection, followed by data-free parameter-space refinement. On the eight-task ViT-B/32 benchmark, the small configuration reaches 89.3% average accuracy (98.7% normalized retention) at 1.24x the storage of one backbone, including that backbone; the medium configuration reaches 89.8% at 1.4x. Removing refinement reduces the medium configuration by 4.3 percentage points, while static-merge plus uniform-recovery controls are 4.7-8.9 points below the small configuration. Additional evaluations cover language and multimodal models at their separately reported budgets. The main results assume known task identity; practical routing and accuracy under larger task counts at fixed capacity remain distinct evaluation questions. Code is available at this https URL.
| Comments: | Camera-ready version accepted at NeurIPS 2026. 38 pages including references, appendices, and checklist. Code: this https URL |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL) |
| Cite as: | arXiv:2605.18904 [cs.LG] |
| (or arXiv:2605.18904v2 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2605.18904 arXiv-issued DOI via DataCite |
Submission history
From: Guodong Du [view email]
[v1]
Sun, 17 May 2026 13:36:53 UTC (1,155 KB)
[v2]
Tue, 20 Oct 2026 11:53:13 UTC (1,040 KB)
来源：arXiv cs.CL · arxiv.org