学习优化器调度中乘性与加性状态耦合的对比：双线性调度实现分布外泛化

Multiplicative vs. Additive State Coupling in Learned Optimizer Schedules

AI 导读

通过展开经典求解器反向传播为 MAX-CUT 连续惩罚松弛学习调度时，状态自适应双线性调度可分布外泛化，在 15 个随机种子上优于未训练基线（t=4.1）；而输入输出头相同的加性消融在分布内与双线性模型统计上无差异，分布外却劣于自身基线（t=-11.7）。该分离在无正则化下出现，并在 30 轮权重衰减扫描中持续存在。结果表明，与分布外迁移相关的是求解器状态如何进入调度，而非状态自适应性本身。

正文 · 原文

View PDF HTML (experimental)

Abstract:We learn schedules for a continuous penalty relaxation of MAX-CUT by backpropagating through an unrolled classical solver. A state-blind schedule generalizes to instances substantially larger than those used for training. A state-adaptive bilinear schedule also generalizes out of distribution, outperforming its untrained baseline across 15 seeds ($t=4.1$). By contrast, an additive ablation with identical inputs and output heads is statistically indistinguishable from the bilinear model in distribution but performs worse than its own baseline out of distribution ($t=-11.7$). This separation appears without regularization and persists across a $30$ round weight-decay sweep: the bilinear advantage remains significant at every tested value, whereas the additive model never recovers. The results in this setting isolate how solver state enters the schedule, rather than state adaptivity alone, as the architectural distinction associated with out-of-distribution transfer.

Subjects:	Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Numerical Analysis (math.NA); Optimization and Control (math.OC)
Cite as:	arXiv:2610.32801 [cs.LG]
	(or arXiv:2610.32801v1 [cs.LG] for this version)
	https://doi.org/10.48550/arXiv.2610.32801 arXiv-issued DOI via DataCite (pending registration)
Journal reference:	Neurips OPT-ML 2026 Workshop

Submission history

From: M. Harper Langston [view email]
[v1] Fri, 23 Oct 2026 15:40:23 UTC (1,177 KB)

来源：arXiv cs.CL · arxiv.org