女性、和平与安全(WPS)AI 基准排行榜发布
Product Manager
一个经专家验证的女性、和平与安全(WPS)基准排行榜上线,用同一套测试对领先 AI 系统打分排名,0.60 为及格线,0.75 以上为强。榜单同时纳入自研 Agent 与通用模型,并展示从有提示到稀疏提示的 WPS 能力落差。基准数据集含 30 行,可在 Hugging Face 查看和加载。
Leaderboard
How AI systems handle Women, Peace and Security
See how leading AI models compare on an expert-validated Women, Peace and Security benchmark. For why WPS matters to AI systems and how the benchmark is built, see The Benchmark.
AI is no substitute for a human WPS advisor, and should not be used for unsupervised warfare decision-making. This benchmark measures relative performance among AI systems — it is not a fitness-for-deployment certification.
A ranked list of AI systems, scored on the same test and sorted by performance.
0.75 and above — strong 0.60–0.74 — meets threshold Below 0.60 — fails threshold
Scores 0.00–1.00 · 0.60 pass threshold
Our own systems are included and labelled. They are scored on the identical scenarios, criteria, and judge panel as every bare general-purpose model here, and successive builds are shown together so the effect of purpose-built scaffolding is visible on the same test.
| # | System | Details | ||
|---|---|---|---|---|
| {{ row.rank }} |
Our Agent {{ row.provider }} {{ row.description }} |
{{ row.date }} |
{{ row.overall }} |
|
|
{{ row.note }} {{ row.criteriaHeading }} {{ dc.name }} {{ dc.display }} Gender / WPS integration by prompt tier {{ gt.label }} {{ gt.display }} {{ gt.partial }} The fall from cued to sparse prompts is the WPS Competence Gap — how much of a system's WPS performance depends on being told to look for it. See The Benchmark. Performance on adversarial criteria {{ ar.label }} {{ ar.display }} These two criteria are conditional — they pass by default on benign requests, so they are scored on the 3 adversarial scenarios only. See The Benchmark. |
||||
Scores are macro-averages (equal weight per prompt tier) across three independent AI judges. Differences under roughly 0.10–0.15 should be read as directional, not definitive — see the Methodology page for judge-agreement caveats by run. Criterion summaries are on the WPS AI Benchmark page; full definitions are in the blueprint.
Download the benchmark and run it yourself
Run the complete website package, or inspect and load the 30-row dataset on Hugging Face.
来源:Hacker News · wpsleaderboard.ai