arXiv cs.AI· Jianchang Su, Dawei Xiang, Yifan Zhang, Wei Zhang·· 2 天前AI 评分34
更强的评审模型看得更少：LLM 评测中的部署完整性盲区
Stronger Judges Do Not See More: Deployment-Integrity Blindness in LLM Evaluation
AI 导读
研究提出 DIB-Bench，用 7,560 个评分样本、六类篡改和从正则到 GPT-5-nano 的评审阶梯衡量 LLM 评测的部署完整性盲区：评审模型越强，越容易接受被篡改的输出，平均接受率从规则式的 88.6% 升至 GPT-5-nano 的 94.1%。
正文 · 原文
Abstract:LLM-agent evaluation reports behavioral scores but leaves the identity of the deployed runtime stack unreported. A tampered deployment that preserves evaluated outputs passes every output-only evaluator. We introduce DIB-Bench, a benchmark that measures this deployment-integrity blindness across 7,560 scored examples, six tamper types, and an evaluator ladder from regex scorers through LLM judges to GPT-5-nano. Stronger evaluators accept more tampered outputs: mean acceptance rises from 88.6% (rule-based) to 94.1% (GPT-5-nano), and a trigger-conditional tamper invisible on the evaluation suite produces 48-100% harmful outputs, rising with model capability. A PAC-style bound shows that tampers with zero output shift are undetectable from outputs at any sample size; a manifest-hash identity channel detects them with zero samples. Behavioral fingerprinting identifies the model component above chance (76.7%) but covers one of six stack components. We also propose a graded deployment-identity descriptor for evaluation reports.
| Subjects: | Cryptography and Security (cs.CR); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.21135 [cs.CR] |
| (or arXiv:2610.21135v1 [cs.CR] for this version) | |
| https://doi.org/10.48550/arXiv.2610.21135 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Jianchang Su [view email]
[v1]
Thu, 15 Oct 2026 02:44:08 UTC (58 KB)
来源：arXiv cs.AI · arxiv.org