跳到正文
Hacker News· frantj·· 8 天前AI 评分38

女性、和平与安全(WPS)AI 基准排行榜发布

Product Manager

AI 导读

一个经专家验证的女性、和平与安全(WPS)基准排行榜上线,用同一套测试对领先 AI 系统打分排名,0.60 为及格线,0.75 以上为强。榜单同时纳入自研 Agent 与通用模型,并展示从有提示到稀疏提示的 WPS 能力落差。基准数据集含 30 行,可在 Hugging Face 查看和加载。

正文 · 原文

Leaderboard

How AI systems handle Women, Peace and Security

See how leading AI models compare on an expert-validated Women, Peace and Security benchmark. For why WPS matters to AI systems and how the benchmark is built, see The Benchmark.

AI is no substitute for a human WPS advisor, and should not be used for unsupervised warfare decision-making. This benchmark measures relative performance among AI systems — it is not a fitness-for-deployment certification.

A ranked list of AI systems, scored on the same test and sorted by performance.

0.75 and above — strong 0.60–0.74 — meets threshold Below 0.60 — fails threshold

Scores 0.00–1.00 · 0.60 pass threshold

Our own systems are included and labelled. They are scored on the identical scenarios, criteria, and judge panel as every bare general-purpose model here, and successive builds are shown together so the effect of purpose-built scaffolding is visible on the same test.

# System Details
{{ row.rank }}

Our Agent

{{ row.provider }}

{{ row.description }}

{{ row.date }}

{{ row.overall }}

{{ row.note }}

{{ row.criteriaHeading }}

{{ dc.name }}

{{ dc.display }}

Gender / WPS integration by prompt tier

{{ gt.label }}

{{ gt.display }} {{ gt.partial }}

The fall from cued to sparse prompts is the WPS Competence Gap — how much of a system's WPS performance depends on being told to look for it. See The Benchmark.

Performance on adversarial criteria

{{ ar.label }}

{{ ar.display }}

These two criteria are conditional — they pass by default on benign requests, so they are scored on the 3 adversarial scenarios only. See The Benchmark.

Scores are macro-averages (equal weight per prompt tier) across three independent AI judges. Differences under roughly 0.10–0.15 should be read as directional, not definitive — see the Methodology page for judge-agreement caveats by run. Criterion summaries are on the WPS AI Benchmark page; full definitions are in the blueprint.

Download the benchmark and run it yourself

Run the complete website package, or inspect and load the 30-row dataset on Hugging Face.

来源:Hacker News · wpsleaderboard.ai