arXiv cs.AI· Hoang Pham Cong, Long D. Nguyen, Ho Viet Duc Luong, Nguyen Tran Minh Nhat, Binh P. Nguyen, Huynh Thi Thanh Binh·· 10 小时前AI 评分29
GUARD:面向冻结多模态模型缺失模态场景的风险受控校正方法
Risk-Controlled Correction for Frozen Multimodal Models under Missing Modalities
AI 导读
针对权重已冻结的多模态模型在模态缺失后无法重训的问题,研究者提出 GUARD(Gated Utility-Aware Recalibration for Deployment),在不改动原模型的前提下通过检索估计候选校正、用部署匹配的标注数据学习何时校正有效,并以保形风险控制校准阈值。
正文 · 原文
Abstract:A deployed multimodal predictor may face missing modalities after its weights can no longer be changed. This raises a question that training-time repair does not answer: what part of the resulting error is still correctable from the information that remains? We show that this part is characterized by the Bayes gap, the difference between the model's prediction and the Bayes prediction given the observed modalities. For cross-entropy and squared error, excess risk measures this gap in the geometry of the loss; ambiguity in the observed modalities cannot be removed. Thus, a badly degraded prediction is not necessarily a correctable one. Estimating the Bayes prediction normally requires labels. However, when a model observing more modalities is conditionally correct, averaging its outputs conditional on the remaining modalities recovers this target without retrieval labels; with the same fixed blend and neighbors, it is no worse in expected cross-entropy or Brier risk than averaging hard labels. Some such assumption is unavoidable: otherwise the target is not identifiable from unlabeled data alone. GUARD (Gated Utility-Aware Recalibration for Deployment) leaves the deployed predictor unchanged. It estimates a candidate correction by retrieval, learns from deployment-matched labeled data when correction is likely to help, and calibrates a threshold on a held-out split by conformal risk control. Under exchangeability, the marginal probability that GUARD applies a correction whose loss increase exceeds a prescribed tolerance is bounded by a preset level, regardless of the learned score. In controlled experiments, correction gain follows headroom rather than raw error; across nine frozen model-dataset pairs spanning six domains, GUARD retains most of blanket correction's gain within harm budgets that blanket correction often exceeds.
| Comments: | Code: this https URL |
| Subjects: | Artificial Intelligence (cs.AI); Machine Learning (stat.ML) |
| Cite as: | arXiv:2611.23886 [cs.AI] |
| (or arXiv:2611.23886v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2611.23886 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Hoang Pham Cong [view email]
[v1]
Wed, 18 Nov 2026 05:39:35 UTC (438 KB)
来源:arXiv cs.AI · arxiv.org