arXiv cs.CL· Tom\'a\v{s} Knap, Ivana Kvapil\'ikov\'a, Jan \v{C}ern\'y, Kl\'ara Bendov\'a, Jarom\'ir \v{S}avelka, Vojt\v{e}ch Pour, Jakub Dr\'apal·· 11 小时前AI 评分26
基于 LLM 的刑事判决信息抽取流水线：面向实证法律研究
Unlocking Criminal Court Decisions: Flexible Information Extraction for Empirical Legal Research
AI 导读
研究者提出一套基于 LLM 的流水线，通过属性探索、文本片段抽取和值归一化构建结构化数据集，并由法律专家迭代完善标注指令。在 DUI 与 DDB 两类交通犯罪上评测，DUI 场景修改指令后抽取准确率从 52% 提升至 70%，且同一模型重复运行的一致性高于其与人工金标注的一致，说明输出稳定不等于正确。该工作已被 JURIX 2026 接收。
正文 · 原文
Abstract:Criminal verdicts contain descriptions of criminal behavior that can support quantitative legal research, but structuring them requires attributes and extraction rules that cannot always be fully specified in advance. We present an LLM-based pipeline that builds a structured dataset through attribute exploration, text span extraction, and value normalization, guided by annotation instructions that legal experts can refine. We evaluate it on two traffic offenses: driving under the influence (DUI) and driving despite a ban (DDB). For DUI, revising the instructions was associated with higher extraction accuracy (from 52% to 70% of correctly extracted values) and higher agreement between human annotators. Repeated runs of the same model agreed with each other more than with the gold annotations, showing that consistent output is not necessarily correct, while disagreement between annotators and between models helped reveal unclear instructions. For DDB, where the instructions defined fixed categories and output formats, agreement was high for most attributes.
| Comments: | Accepted for publication in the proceedings of the 39th International Conference on Legal Knowledge and Information Systems (JURIX 2026) |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.03051 [cs.CL] |
| (or arXiv:2611.03051v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2611.03051 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Tomas Knap [view email]
[v1]
Mon, 2 Nov 2026 12:46:53 UTC (1,380 KB)
来源：arXiv cs.CL · arxiv.org