arXiv cs.AI· Tian Bai, Ying Jin·· 13 小时前AI 评分18
OptCS：数据驱动评分构建后仍可控制 FDR 的共形选择框架
Optimized Conformal Selection: Powerful Selective Inference After Data-Dependent Score Construction
AI 导读
研究者提出 Optimized Conformal Selection（OptCS）框架，允许在灵活的数据驱动评分构建之后仍进行有效的多重检验，并在大量数据复用下保持有限样本 FDR 控制。该框架给出三种控制 FDR 的流程：从多个候选中挑选强评分函数、不做样本划分用全部标注数据训练评分、以及结合全样本训练与模型选择。方法在模拟研究及药物发现和大语言模型的实际应用中验证了有效性。
正文 · 原文
Abstract:We study data-driven score construction in conformal selection, which uses conformal p-values and multiple testing to select "interesting" instances with large unobserved labels from a pool of unlabeled data, while controlling the FDR in finite sample. While existing solutions require the score function to be constructed independent of the calibration and test data, it is desirable to select or construct powerful conformity score in a data-driven manner and to leverage all labeled data for score training when presented with many score choices and limited labeled data. We propose Optimized Conformal Selection (OptCS), a general framework that allows valid multiple testing after flexible data-driven score construction. We introduce general conditions under which OptCS constructs valid conformal p-values despite substantial data reuse and handles complex p-value dependencies to maintain finite-sample FDR control. We instantiate this general recipe to propose three FDR-controlling procedures, each optimizing the models differently: (i) selecting a powerful score function among multiple candidates, (ii) training the score with all labeled data without sample splitting, and (iii) combining full-sample training and model selection. We demonstrate the efficacy of our methods via simulation studies and real applications in drug discovery and large language models.
| Subjects: | Methodology (stat.ME); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Machine Learning (stat.ML) |
| Cite as: | arXiv:2411.17983 [stat.ME] |
| (or arXiv:2411.17983v2 [stat.ME] for this version) | |
| https://doi.org/10.48550/arXiv.2411.17983 arXiv-issued DOI via DataCite |
Submission history
From: Tian Bai [view email]
[v1]
Wed, 27 Nov 2024 01:40:50 UTC (1,972 KB)
[v2]
Wed, 21 Oct 2026 06:51:07 UTC (2,383 KB)
来源：arXiv cs.AI · arxiv.org