廉价验证器 Jev 核查 RAG 声明准确率逼近 GPT-6 Astra，成本低 187 倍
廉价验证器 Jev 核查 RAG 声明准确率逼近 GPT-6 Astra
论文在 495 条人工标注声明上对比了仅输出标签和概率的廉价验证器 Jev 与 GPT-6 Astra，Jev 准确率与 Astra 相差 0.6 个百分点以内，成本低 187 倍。但 Jev 放行了 23.2% 的无依据声明，高于 Astra 的 14.1%。级联方案虽能降低 Astra 费用，但随机分流同样比例的声明也能省下差不多的钱。
Rohan Paul· @rohanpaul_ai · X·2026-10-27 18:24· 2 小时前AI 评分40
AI 导读
论文测试了仅输出标签和概率的 Jev 与 GPT-6 Astra 在 495 条人工标注声明上的核查表现，Jev 准确率与 Astra 相差 0.6 个百分点以内，成本低 187 倍。但 Jev 放行了 23.2% 的无依据声明，高于 Astra 的 14.1%。级联方案虽降低 Astra 费用，但随机分流同样比例的声明也能省下差不多的钱。
正文 · 原文
This paper finds that a cheap verifier can match a frontier LLM's accuracy at checking RAG claims, while letting more unsupported claims slip through.
RAG pipelines keep asking whether evidence supports a claim, a simple yes or no. Most teams get that answer from a text-generating LLM at generation prices.
They tested Jev, which returns only a label and a probability, against GPT-6 Astra on 495 human-labelled claims. They also built a cascade where Jev answers confident claims and passes the rest to Astra.
Jev landed within 0.6 points of Astra's accuracy at 187 times lower cost, but it approved 23.2% of unsupported claims against Astra's 14.1%. The cascade also cut Astra's bill, yet sending the same share of claims to Astra at random saved about as much.
Before picking a claim checker for a RAG product, measure how often it approves unsupported claims, since headline accuracy hides that.
来源：Rohan Paul · x.com
来源：RadarAI · aihot.news