arXiv cs.CL· Aswathy Velutharambath, Neele Falk, Sofie Labat, Tarun Tater, Amelie Wuehrl·· 2 天前AI 评分29
众包能挺过 LLM 时代吗？一项关于人类数据收集的社区调查
Can Crowdsourcing Survive the LLM Era? A Community Survey on Human Data Collection
AI 导读
一项针对 155 名 NLP 及相关领域研究者的调查显示，44% 的受访者在自己的众包数据中观察到 LLM 使用痕迹，其中 93% 早有预期，但半数不知该采取何种防范措施。最常见的检测手段是独特的文本风格模式和异常快速的完成时间。调查表明研究社区已意识到该问题并采取措施，但现有努力仍不足以完全应对。
正文 · 原文
Abstract:The widespread use of Large Language Models (LLMs) as writing tools challenges the validity of crowdsourced data, as crowdworkers may outsource tasks to models. To better understand how this is addressed, we surveyed 155 researchers in NLP and related disciplines about their experiences and opinions on collecting free-text responses via crowdsourcing. This paper provides an overview of practitioners' challenges, mitigation strategies, and the foreseen implications on data quality. 44% of respondents reported observing LLM usage in their crowdsourced data. While 93% of them had anticipated this, half were unsure what precautions to take. The most prevalent detection strategies are distinctive textual style patterns and unusually fast completion times. Overall, survey responses show that the research community is aware of the problem and taking measures, but existing efforts remain insufficient to fully address it. Finally, we derive a set of considerations to guide future crowdsourced free-text data collection in the era of LLMs.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2606.04924 [cs.CL] |
| (or arXiv:2606.04924v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2606.04924 arXiv-issued DOI via DataCite |
Submission history
From: Aswathy Velutharambath [view email]
[v1]
Wed, 3 Jun 2026 14:18:27 UTC (538 KB)
[v2]
Thu, 15 Oct 2026 09:05:51 UTC (716 KB)
来源：arXiv cs.CL · arxiv.org