arXiv cs.CL· Ramya Namuduri, Manya Wadhwa, Anshun Asher Zheng, Greg Durrett, Junyi Jessy Li·· 13 小时前AI 评分31
GENIE：面向任务特定特征的细粒度新颖性度量指标
GENIE: A Fine-Grained Measure for Novelty
AI 导读
研究者提出细粒度评估指标 GENIE，按任务特定特征衡量模型回复相对于回复群体的新颖性。与 GENIE 不同，整体性指标难以捕捉新颖性的高维特性，也无法说明其针对哪些属性。GENIE 还被用于评估针对创造力的缓解方法在哪些方面能提升新颖性，该工作已被 EMNLP 主会接收。
正文 · 原文
Abstract:Large Language Models have consistently demonstrated a lack of creativity and diversity across tasks. Prior work has focused on addressing whether models are capable of generating creative outputs. Here, we aim to consider novelty and investigate what makes model-generated content novel or not novel in a task-specific manner. We propose a fine-grained evaluation metric GENIE to measure the novelty of responses along task-specific features with respect to a population of responses. We show that unlike GENIE, holistic metrics struggle to capture the high-dimensionality of novelty and do not provide insight on which properties they target. Finally, we use GENIE to measure the effectiveness of mitigation methods that address creativity to better understand where these methods can improve novelty.
| Comments: | Accepted to EMNLP Main Conference (2026) |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2606.12790 [cs.CL] |
| (or arXiv:2606.12790v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2606.12790 arXiv-issued DOI via DataCite |
Submission history
From: Ramya Namuduri [view email]
[v1]
Thu, 11 Jun 2026 01:14:10 UTC (3,416 KB)
[v2]
Wed, 21 Oct 2026 17:23:27 UTC (3,407 KB)
来源：arXiv cs.CL · arxiv.org