arXiv cs.AI· Aditya Nagarsekar, M P Ashish Bhat, Aadi Nesarkar, Vrishti Godhwani, Rahul Yedida, Aditya Challa, Danda Sravan, Snehanshu Saha·· 13 小时前AI 评分18
泛化信号未必是模型选择信号：Hessian 几何指标在分布外任务上的失效
A Generalisation Signal Need Not Be a Model-Selection Signal
AI 导读
研究测试了以 Hessian 范数为动机的前向代理指标及常见 Hessian 度量，在分子性质、蛋白质适应度和药物反应任务上能否用于模型选择。结果显示，验证 Spearman 相关性恶化时几何指标并未变得更有用，该代理虽在多数任务上与泛化差距相关，却无法可靠识别部署最优模型，低几何分数甚至可能偏向坍缩的预测器。
正文 · 原文
Abstract:Model selection in computational biology often relies on validation data drawn from the training regime, even when deployment lies outside it. When validation no longer preserves which model is best, a natural alternative is to rank candidates using properties of the trained network itself. We test this idea using a novel, forward-only proxy motivated by the norm of the Hessian, alongside common Hessian measures, across molecular property, protein fitness, and drug-response tasks. Contrary to our hypothesis, geometry does not become more useful as validation Spearman correlation deteriorates: augmenting validation helps some shifts but significantly harms others. More surprisingly, the proxy still correlates with generalisation gap on most tasks even when Hessian trace and top-eigenvalue relationships are weak or reversed, yet this signal does not reliably identify the deployment-best model. A curvature bound need not preserve cross-model rankings, and low geometric scores can even favour collapsed predictors. Thus, a generalisation signal need not be a model-selection signal.
| Comments: | Accepted (poster) at the NeurIPS 2026 Workshop "I Can't Believe It's Not Better: Failure Modes of AI in Biology" (ICBINB-BIO). 24 pages, 4 figures, 22 tables. Code: this https URL |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2609.39099 [cs.LG] |
| (or arXiv:2609.39099v2 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2609.39099 arXiv-issued DOI via DataCite |
Submission history
From: Aditya Nagarsekar [view email]
[v1]
Wed, 30 Sep 2026 06:36:54 UTC (157 KB)
[v2]
Wed, 21 Oct 2026 15:43:21 UTC (197 KB)
来源：arXiv cs.AI · arxiv.org