arXiv cs.AI· Jo\~ao Carlos Santana Silveira, Caio Von Rondow Morais, Frederico Pedrosa, Lucas N. Ferreira·· 4 小时前AI 评分20
用生成式心理测量学评估音乐生成模型的评测方法
Evaluating the Evaluation of Music Generation Models with Generative Psychometrics
AI 导读
研究梳理了 ISMIR 2020-2025 年音乐生成论文中的主观评测方法,用 LLM 加人工校验从论文方法描述中提取 284 个题项并标准化为上下文嵌入向量。
正文 · 原文
Abstract:Subjective user studies are one of the major methods for evaluating music generation models. However, subjective evaluation protocols vary widely and are rarely specified in enough detail to be reproduced, hindering comparison across studies. In this paper, we map subjective evaluation methods in ISMIR music generation papers from 2020-2025 and apply generative psychometrics to assess the stability of their items. We extract 284 items from the papers' methodological descriptions, using an LLM and manual validation to standardize them. We represent these items as contextual embeddings and apply Unique Variable Analysis to remove redundant, locally dependent items and Bootstrap Exploratory Graph Analysis to estimate their latent dimensions. Results show the network converges onto a taxonomy of six stable quality dimensions: Rhythm, Melody, Audio, Composition, Structure, and Lyrics, pointing to a unified methodology for conducting user studies to evaluate music generation models.
| Comments: | Accepted to the 2nd Latin American Music Information Retrieval (LAMIR) workshop |
| Subjects: | Sound (cs.SD); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.21809 [cs.SD] |
| (or arXiv:2611.21809v1 [cs.SD] for this version) | |
| https://doi.org/10.48550/arXiv.2611.21809 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Lucas N. Ferreira [view email]
[v1]
Mon, 16 Nov 2026 19:23:47 UTC (1,894 KB)
来源:arXiv cs.AI · arxiv.org