arXiv cs.AI· Shenghui Li, Thiemo voigt·· 11 小时前AI 评分24
预算受限下的 LoRA：面向适配器集群的功能感知事后压缩方法 FRA
LoRA on a Budget: Function-Aware Post-Hoc Compression for Adapter Fleets
AI 导读
研究者提出 Functional Rank Allocation（FRA），一种无需标注的事后压缩方法，用于在共享 rank 预算下为 LoRA 适配器集群分配各层 rank。FRA 通过端到端测量单模块截断构建功能 rank-失真前沿，再用动态规划搜索并复测候选，最后做精确 minimax 分配。在三个公开集群共 78 个适配器上，FRA 在受限预算下优于统一谱压缩的最差情况与尾部效用。
正文 · 原文
Abstract:Low-Rank Adaptation (LoRA) has become a key mechanism for efficiently specializing foundation models into many task- and user-specific variants. Serving many LoRA adapters under a shared resource budget turns post-hoc compression into a two-level allocation problem: rank must be distributed across layers within each adapter and across adapters within the fleet. Although trained LoRAs contain substantial spectral redundancy, spectral reconstruction alone does not determine how much compression an adapter can functionally tolerate. We formulate fleet compression through a functional rank-distortion frontier for each adapter, followed by fleet-level minimax allocation under a shared rank budget. Given exact frontiers, this decomposition is equivalent to joint optimization over all layer ranks. We introduce Functional Rank Allocation (FRA), an unlabeled method for constructing practical frontiers. FRA measures isolated module truncations end to end, uses these measurements as an additive search surrogate solved by dynamic programming, remeasures every complete candidate, and takes the lower-risk envelope with a complementary spectral proposal before exact minimax allocation over the constructed frontiers. Across 78 adapters from three public fleets, FRA improves worst-case and tail utility over uniform spectral compression in constrained-budget regimes. Fixed-budget probes show that isolated costs rank candidates well, but their calibration shifts across proposal shapes; end-to-end verification and the spectral proposal therefore limit functional-search error.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.36386 [cs.AI] |
| (or arXiv:2610.36386v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.36386 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Shenghui Li [view email]
[v1]
Mon, 26 Oct 2026 23:02:12 UTC (50 KB)
来源：arXiv cs.AI · arxiv.org