跳到正文
arXiv cs.CY· Gregory M. Dickinson·· 9 小时前AI 评分31

别再删停用词:继承的预处理默认设置如何扭曲法律文本数据研究

Stop Removing Stopwords: How an Inherited Preprocessing Default Distorts Legal Text-as-Data

AI 导读

一项针对美国最高法院判决文本的研究发现,停用词删除这一继承自信息检索领域的预处理默认设置,在意识形态方向(无删除基线 F1 约 0.68)与法律类型(约 0.92)两个二分类任务上均无收益,通用停用词表在全部十二项留出测试中甚至低于不删除基线。

正文 · 原文

View PDF

Abstract:Empirical legal scholarship increasingly treats judicial text as data, and much of it still runs on sparse, interpretable pipelines -- TF-IDF features and linear classifiers -- because the textual feature is often the object of study rather than a means to prediction. Yet these pipelines inherit preprocessing defaults from mid-century information retrieval that were never validated against classification accuracy. The most entrenched of these is stopword removal. This study introduces an exhaustive single word ablation that measures the effect of a preprocessing step directly against the downstream objective, and applies it to stopword removal as the hardest case to dislodge. Matching Supreme Court Database labels to Caselaw Access Project opinion texts, the study examines two binary tasks that bracket F1 headroom, ideological direction (no-removal baseline F1 ~0.68) and constitutional versus non-constitutional law type (~0.92), across 7,668 and 7,001 opinions. For each task, the ablation removes each of about 18,500 candidate words in turn, and a task-specific stoplist is built from the measurements. Common generic stoplists fall below the no-removal baseline in all twelve held-out tests. The task-specific stoplists move held-out F1 by +0.0023 (95% CI [-0.0124, +0.0170]) on ideology and by +0.0001 ([-0.0082, +0.0085]) on law type. Neither task shows a detectable benefit from removal, and supplemental analysis finds that word-level statistics predict removal effects poorly, because true removal effects differ by less than the measurement can register, leaving little to predict. The method generalizes to any inherited preprocessing default; the result is a caution specific to interpretable legal text-as-data, where a step that reshapes which features a model sees can distort the doctrinal and ideological signal the research is meant to recover. The burden of proof sits with removal.
Subjects: Computation and Language (cs.CL); Computers and Society (cs.CY); Information Retrieval (cs.IR); Machine Learning (cs.LG)
MSC classes: 68T50 (Primary) 68P20, 62H30 (Secondary)
ACM classes: I.2.7; H.3.1; I.5.2; H.3.3; I.2.6; J.1
Cite as: arXiv:2609.19153 [cs.CL]
  (or arXiv:2609.19153v3 [cs.CL] for this version)
  https://doi.org/10.48550/arXiv.2609.19153

arXiv-issued DOI via DataCite

Submission history

From: Gregory Dickinson [view email]
[v1] Tue, 21 Jul 2026 14:55:19 UTC (599 KB)
[v2] Sun, 4 Oct 2026 20:44:08 UTC (936 KB)
[v3] Thu, 19 Nov 2026 17:15:14 UTC (1,836 KB)

来源:arXiv cs.CY · arxiv.org