Benchmark Radar：面向 AI 基准与评测的实时数据库和搜索引擎
Benchmark Radar: A Living Database and Search Engine for AI Benchmarks and Evaluation
Benchmark Radar 是一个面向 AI 基准检索与发现的实时数据库和搜索引擎，覆盖 LLM 评测、智能体与工具使用、编码、推理、安全及领域评测。系统每日从 37 个来源（13 个直连、24 个第一方研究 feed）发现基准论文、仓库、数据集与发布，目录含 1,283 条来源记录、790 条记录上的 12,916 个数值观测。
Authors:Koutian Wu, Wenjun Zhang, Junjie Zhou, Pengqian Han, Qiming Bao, Ergan Shang, Jiayu Wang, Junkai Wang, Wanghan Xu, Songyuanyi Lu, Yixuan Li, Rudy Mizrahi Celekli, Lazizbek Ravshanov, Yaoliang Bian, Xudong Han, Lin Shi
Abstract:Benchmark researchers and developers of large language models (LLMs) and other AI systems need to find relevant evaluations, locate their benchmark datasets and code, and understand the settings behind reported scores. We present Benchmark Radar, a living database and search engine for retrieval and discovery of AI benchmarks, covering LLM evaluation, agentic and tool-use benchmarks, coding, reasoning, safety, and domain-specific evaluations. The system combines daily discovery of benchmark papers, repositories, datasets, and releases with a searchable benchmark catalog, mentions in model cards and technical reports, and score histories. It retains source identities and citations so readers can inspect candidate benchmarks and their evaluation evidence. Daily discovery draws on 37 sources: 13 direct connectors and 24 first-party research and engineering feeds. The catalog contains 1,283 source records drawn from 4 benchmark catalogs and 12,916 numeric observations on 790 records. We describe collection and retrieval, audit the full catalog, and examine benchmark saturation, adoption trends, and the limits of score comparisons. A worked example walks through a complete prior-art search, showing how to query the catalog and inspect benchmark evidence when designing a new evaluation. We release the web dashboard with a benchmark leaderboard, a Pareto frontier view of score against measured use, saturation and trend views, daily feeds, downloadable evidence, a command-line interface (CLI) for offline queries, and reproducible analysis.
| Comments: | Code: this https URL, Project site: this https URL |
| Subjects: | Artificial Intelligence (cs.AI); Information Retrieval (cs.IR) |
| Cite as: | arXiv:2609.11115 [cs.AI] |
| (or arXiv:2609.11115v4 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2609.11115 arXiv-issued DOI via DataCite |
Submission history
From: Koutian Wu [view email]
[v1]
Thu, 10 Sep 2026 05:47:22 UTC (4,314 KB)
[v2]
Sun, 13 Sep 2026 07:02:41 UTC (4,315 KB)
[v3]
Tue, 22 Sep 2026 03:36:21 UTC (4,316 KB)
[v4]
Tue, 27 Oct 2026 05:50:03 UTC (4,050 KB)
来源：arXiv cs.AI · arxiv.org