arXiv cs.CL· Chantal Shaib, Tuhin Chakrabarty, Diego Garcia-Olano, Byron C. Wallace·· 12 小时前AI 评分31
如何衡量文本中的 AI“Slop”：一套可解释的评估维度框架
Measuring AI "Slop" in Text
AI 导读
研究者通过与 NLP、写作和哲学领域专家访谈，构建了 AI“slop”（低质量 AI 生成文本）的分类体系，并提出一组可解释的文本评估维度。基于 span 级标注，研究发现二元“slop”判断带有一定主观性，但与连贯性、相关性等潜在维度相关。该框架可用于检测任务和二元偏好任务，但完全自动化、可规模化的方法仍是未解难题。
正文 · 原文
Abstract:AI "slop" is an increasingly popular term used to describe low-quality AI-generated text, but there is currently no agreed upon definition of this term nor a means to measure its occurrence. In this work, we develop a taxonomy of "slop" through interviews with experts in NLP, writing, and philosophy, and propose a set of interpretable dimensions for its assessment in text. Through span-level annotation, we find that binary "slop" judgments are (somewhat) subjective, but such determinations nonetheless correlate with latent dimensions such as coherence and relevance. Our framework can be used to examine AI-generated text in both detection and binary preference tasks, potentially offering new insights into the linguistic and stylistic factors that contribute to quality judgments. We highlight that fully automated and scalable methods remain an open challenge.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2509.19163 [cs.CL] |
| (or arXiv:2509.19163v3 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2509.19163 arXiv-issued DOI via DataCite |
Submission history
From: Chantal Shaib [view email]
[v1]
Tue, 23 Sep 2025 15:41:19 UTC (882 KB)
[v2]
Sat, 24 Jan 2026 04:46:30 UTC (887 KB)
[v3]
Fri, 16 Oct 2026 17:05:27 UTC (924 KB)
来源：arXiv cs.CL · arxiv.org