arXiv cs.AI· Mateusz Praski, Jakub Adamczyk, Wojciech Czech·· 11 小时前AI 评分37
预训练分子嵌入模型基准测试：28 个模型在 25 个数据集上未能超越 ECFP 指纹
Benchmarking Pretrained Molecular Embedding Models For Molecular Representation Learning
AI 导读
一项研究在 25 个分类数据集上对比了 28 个预训练分子嵌入模型，采用分层贝叶斯 Bradley-Terry 统计检验，结果显示除 CheMeleon 外所有神经网络模型均未能超越经典 ECFP count 指纹，CheMeleon 是唯一被评为更优的模型。
正文 · 原文
Abstract:Pretrained molecular embedding models spanning graph neural networks, text transformers, and hybrid architectures have been widely used in drug discovery, yet no large-scale, rigorous comparison of their representations existed. We evaluated 28 models across 25 classification datasets under a controlled framework using hierarchical Bayesian Bradley-Terry statistical testing. All neural models except CheMeleon failed to outperform the classical ECFP count fingerprint; CheMeleon was the only model rated as better. However, further testing on external validation datasets, based on scaffold cross-validation and a Bayesian hierarchical correlated t-test, revealed no definite advantage of either embedding method. As a practical takeaway, we recommend evaluating both ECFP fingerprints and CheMeleon embeddings in novel QSAR/QSPR studies. Overall, in the context of all neural networks evaluated, these results also challenge the perceived progress in molecular representation learning and highlight the need for rigorous evaluation standards in the field.
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2508.06199 [cs.LG] |
| (or arXiv:2508.06199v5 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2508.06199 arXiv-issued DOI via DataCite |
Submission history
From: Mateusz Praski [view email]
[v1]
Fri, 8 Aug 2025 10:29:24 UTC (2,277 KB)
[v2]
Wed, 13 Aug 2025 06:01:26 UTC (2,277 KB)
[v3]
Sun, 14 Sep 2025 11:09:51 UTC (2,277 KB)
[v4]
Mon, 23 Feb 2026 14:41:11 UTC (2,286 KB)
[v5]
Tue, 3 Nov 2026 20:20:40 UTC (3,820 KB)
来源：arXiv cs.AI · arxiv.org