arXiv cs.CY· Ahmed M. Mahran·· 13 小时前AI 评分44
超越感知之外：AI 评估与监督局限的分区理论
What Lies Beyond Perception: A Partition Theory of the Limits of AI Evaluation and Oversight
AI 导读
研究提出评估的“分区理论”：智能体的判别力是把输入划分为无法区分的类别，判别力是能力的必要条件而非充分条件，因此评估只能给出能力下限、无法证明上限。作者证明，若受测者的分区比评估者更细，它就能通过评估者能评分的所有测试；据此定义的 echo test 对评估者可评分的测试是完备的。该结论可扩展到多监控器集成与模型可控通道，并解释评审分歧与 sandbagging 的已有发现。
正文 · 原文
Abstract:Replacing the judge can move a benchmark score, and models can be trained to hide capabilities. Both put the evaluator in question: what can it certify? We develop a partition theory of evaluation from one principle: judgement is bounded by perception. An agent's discrimination is the partition of its inputs into classes it cannot tell apart. Discrimination is necessary for capability but not sufficient, so bounds on it bear directly on whether evaluation can show that a model lacks an ability. We prove a one-sided bound. An examinee whose partition refines its evaluator's has the discrimination to pass every test the evaluator can grade, provided each input has an acceptable reply. We define the echo test, which accepts any reply the evaluator perceives as identical to the input, and show that it is complete for the tests the evaluator can grade. Passing therefore certifies at most a floor on discrimination, never a ceiling, and only a sharper evaluator can raise it. Used to detect concealment, echo-test failure points the wrong way: under exhaustive probing it flags every honestly weaker examinee and no concealer that presents as the evaluator's peer. Exact imitation of a weaker model is undetectable by any behavioural evaluator. We extend the bound to ensembles of monitors and to channels a model controls, explain published findings on judge disagreement and sandbagging, and derive testable predictions. A safety case that argues from behavioural evaluation alone that a model lacks an ability must assume the model's cooperation.
| Comments: | 36 pages (23 main text, then references and appendices), 6 figures, 2 tables |
| Subjects: | Artificial Intelligence (cs.AI); Computers and Society (cs.CY); Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.30803 [cs.AI] |
| (or arXiv:2610.30803v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.30803 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Ahmed Mahran [view email]
[v1]
Thu, 22 Oct 2026 06:28:29 UTC (47 KB)
来源：arXiv cs.CY · arxiv.org