arXiv cs.AI· Zhirong Su, Tian Zhang·· 4 小时前AI 评分24
五层框架审计基准测量:以 UAV 意图识别为例拆解编码、可解码性与使用
Encoding, Decodability, and Use: A Five-Layer Framework for Benchmark Measurement Audits with Evidence from UAV Intent Recognition
AI 导读
研究者提出并在 AeroIntent 上实例化五层框架,将基准准确率拆为观察、表征、解码、使用与行为五层,区分编码、可解码性与使用。观察层在 20 个 Drone 程序中 7 个不可靠,径向速度直接视觉相关项在 0.26% 尺度变化下触底;解码层多特征 Bonferroni 阈值下 MDE 为 |r|=0.599,而最大样本外相关仅 r=+0.080、power 为 0.00。
正文 · 原文
Abstract:Benchmark accuracy is routinely read as evidence that a model understands the construct the benchmark measures. We propose and instantiate on AeroIntent a five-layer framework that decomposes that reading into observation, representation, decoding, use, and behaviour. The framework separates encoding, decodability, and use as distinct levels. In the instantiation, the observation layer is unreliable on 7 of the 20 Drone programs and floors the direct visual correlate of radial velocity at a 0.26\% scale change. The representation layer exposes two task-designed RGB features with nominal signals. RAFT flow radial bias reaches $r=+0.257$ ($p=0.030$), and bounding-box optical-flow divergence reaches $r=+0.203$ ($p=0.029$). No multi-feature readout gives a fold-mean test correlation above the cluster minimum detectable effect (MDE), and the largest positive global out-of-sample correlation is $r=+0.080$. The decoding layer sets the MDE at $|r|=0.599$ at the multi-feature Bonferroni threshold ($\alpha=0.00714$, power 0.80), and the observed $r=+0.080$ has power 0.00. Two injection experiments establish the MDE's condition from both sides: aligned injections are detected at or above the simulated power, and orthogonal injections are not detected. The use layer finds accuracy consistent with program memory (program-identity 0.937/0.597/0.210) and multi-region static cues. The behaviour layer is consistent with accuracy not requiring a radial-velocity readout (appearance 0.6552 vs bounding-box trajectory 0.4080, cluster-level paired $\Delta=+0.2471$, $p=0.0132$).
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.22434 [cs.LG] |
| (or arXiv:2611.22434v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2611.22434 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Tian Zhang [view email]
[v1]
Tue, 17 Nov 2026 09:08:17 UTC (120 KB)
来源:arXiv cs.AI · arxiv.org