arXiv cs.AI· Daoan Zhang, Qinbo Bai, Dehai Min, Xingyu Mu, Ziyi Chen, Jianing Yin·· 10 小时前AI 评分42
Veritas：面向智能体任务的验证器生成框架
Veritas: A Verifier Generation Harness for Agent Tasks
AI 导读
Veritas 是一个自动生成验证器的框架，通过证据头、领域规则库和编写原则三种机制生成验证器，其中编写原则部分由自我改进循环优化。在四个按执行评分的基准上，固定模型时 Veritas 生成的验证器与官方判定的一致度优于其他七个框架；在两个成对比较基准上达到或接近最佳。在另外三个模型家族中，有两个家族的表现优于厂商自带框架，其机制还在三个框架的 12 个执行评分组合中改善了 9 个。
正文 · 原文
Abstract:The effectiveness of a self-improving system relies heavily on the quality of its verifier, which judges the outcome of each improvement round. However, the verifiers of most tasks are written by hand, which is expensive and does not scale to tasks that come without one. We propose Veritas, a harness that generates verifiers automatically. Unlike general-purpose agent harnesses, Veritas adds three mechanisms for verifier generation: an evidence header stating what the verifier will observe, a library of domain rules that tasks leave unstated, and authoring principles, partly refined by a self-improvement loop. We evaluate on six benchmarks: four scored by execution, whose official verifiers we first certify, and two that compare pairs of responses. With the model fixed, the verifiers Veritas generates agree better with the official verdicts than those of seven other harnesses on all four execution-scored benchmarks. On the two pairwise benchmarks they are at or near the best. They also agree better than those of the vendor's own harness for two of three other model families. Its mechanisms also improve 9 of 12 harness-benchmark pairs scored by execution across three harnesses.
| Comments: | 31 pages, 1 figure |
| Subjects: | Artificial Intelligence (cs.AI); Cryptography and Security (cs.CR) |
| Cite as: | arXiv:2611.13336 [cs.AI] |
| (or arXiv:2611.13336v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2611.13336 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Qinbo Bai [view email]
[v1]
Mon, 9 Nov 2026 23:01:40 UTC (270 KB)
来源：arXiv cs.AI · arxiv.org