arXiv cs.CL· Noureddine Khaous (LLF - UMR7110, UPCit\'e), Guillaume Wisniewski (LLF - UMR7110, UPCit\'e)·· 6 天前AI 评分22
更好的度量救不了你：比较什么比怎么比较更重要
A Better Metric Will Not Save You: What You Compare Beats How You Compare It
AI 导读
表征相似性度量（RSA、CKA、Procrustes）的选择取决于其视为无关的变换，但所有度量都把条件集合当作既定输入，而该集合由"纳入哪些输入"和"如何分组"两个决策在矩阵生成前就已固定。
正文 · 原文
Abstract:Representational similarity measures (RSA, CKA, Procrustes) are chosen for the transformations they treat as irrelevant, and all of them take the set of conditions as given. That set is fixed by two decisions, which inputs are admitted and how they are grouped, both taken before the matrix a measure operates on exists: no invariance property reaches them. We measure what they are worth where the arrangement the conditions ought to have is known independently of the systems compared, the acoustic organisation of vowels, across fifteen languages and twenty-seven representations. Changing which categories are compared, at a fixed number of them, moves the gap between two representations by more than twice the effect being measured, while changing how many moves it not at all. In this setting rankings of models are identified under these choices, and rankings across languages are not.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.13476 [cs.CL] |
| (or arXiv:2610.13476v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.13476 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Guillaume Wisniewski [view email] [via CCSD proxy]
[v1]
Fri, 9 Oct 2026 12:11:53 UTC (101 KB)
来源：arXiv cs.CL · arxiv.org