arXiv cs.CL· Wenliang Liang, Zhiyuan Ning, Haolin Chen, Yuanchun Zhou, Wenjuan Cui, Yi Du·· 11 小时前AI 评分31
ProSearch：实验科学中多约束协议检索的基准测试
ProSearch: Benchmarking Multi-Constraint Protocol Retrieval in Experimental Science
AI 导读
ProSearch 是面向实验科学的多约束协议检索基准，覆盖 6 个实验科学领域、60 个实验领域及其依赖关系，共含 1,515 条查询，每条查询带 3-5 个显式约束（含数值与否定约束）。在 19 个代表性检索器上评测，最佳检索器平均 nDCG@10 仅 0.290，加入重排序后提升至 0.474。结构化诊断分析显示显式约束匹配是强排序信号，联合约束绑定是主要瓶颈。
正文 · 原文
Abstract:In AI for Science scenarios, experimental design and method reproduction require retrieving protocols that satisfy multiple interdependent constraints, yet existing benchmarks lack systematic evaluation of this capability. To address this gap, we introduce ProSearch, a benchmark for multi-constraint protocol retrieval in experimental science. ProSearch has three key features: (1) realism, with queries grounded in real experimental records and expert-defined templates; (2) complexity, with 60 experimental fields and their dependencies; and (3) diagnosticity, with 3-5 explicit constraints per query, including numerical and negation constraints. We construct ProSearch through LLM-based extraction, data-driven constraint selection, controlled query generation, and evidence-based relevance validation. The resulting benchmark contains 1,515 queries across six experimental science domains. We conduct extensive experiments on ProSearch with 19 representative retrievers and complementary analyses. The best-performing retriever achieves an average nDCG@10 of only 0.290, which improves to 0.474 with reranking. Structured diagnostic analysis shows that explicit constraint matching provides a strong ranking signal, while manual error analysis identifies joint constraint binding as a key bottleneck. ProSearch provides a testbed for systematically evaluating scientific retrieval systems.
| Comments: | Accepted to NeurIPS 2026 (Evaluations and Datasets Track). 48 pages, 4 figures. Wenliang Liang and Zhiyuan Ning contributed equally. Code: this https URL. Data: this https URL |
| Subjects: | Information Retrieval (cs.IR); Artificial Intelligence (cs.AI); Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.41396 [cs.IR] |
| (or arXiv:2610.41396v1 [cs.IR] for this version) | |
| https://doi.org/10.48550/arXiv.2610.41396 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Zhiyuan Ning [view email]
[v1]
Fri, 30 Oct 2026 08:06:24 UTC (245 KB)
来源：arXiv cs.CL · arxiv.org