arXiv cs.CL· Shuyuan Zheng, Yun-seo Lee, Xing Gao, Jialu Wu, Jiabao Hu, Katrina Qiyao Wang, Xinbo Lin, Qianying Liu, Chuan Xiao, Tao Huang·· 3 天前AI 评分23
LLM 如何理解法哲学？LePhilBench 诊断大语言模型的法哲学推理能力
How Do LLMs Understand Legal Philosophy? Diagnosing Legal-Philosophical Reasoning in LLMs
AI 导读
研究者提出 LePhilBench，用于评估 LLM 的法哲学推理能力，包含两项基准任务与一个数据集。其测试集收录 220 份美国最高法院判决意见，由法学教授设计 14 个法哲学标注问题，3 名受过法学训练的标注者完成全部标注，共产生 9,240 个法哲学标签。
正文 · 原文
Abstract:Legal reasoning is a fundamental capability for large language models (LLMs) to complete legal AI tasks such as legal judgment prediction. In legal practice, this reasoning is shaped by legal philosophy: a judge's view of the nature and sources of law guides how legal materials are interpreted and applied, so judges working from the same statutes and precedents can reach different judgments. Existing research has largely overlooked whether LLMs can understand legal philosophy and apply it in legal reasoning, an ability we call \textit{legal-philosophical reasoning}. To fill this gap, we introduce LePhilBench, a benchmark for evaluating the legal-philosophical reasoning of LLMs that comprises two benchmarking tasks and a dataset. The two tasks evaluate how well LLMs understand legal philosophy and how well they apply it in legal reasoning, respectively. The dataset is manually and meticulously constructed. Its test set contains 220 U.S. Supreme Court opinions. A law professor designed fourteen annotation questions on legal philosophy, and three law-trained annotators with formal training in jurisprudence answered all fourteen questions for every opinion, yielding 9,240 legal-philosophical labels. LePhilBench takes the first step toward exploring legal-philosophical reasoning in LLMs.
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.19427 [cs.CL] |
| (or arXiv:2610.19427v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.19427 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Shuyuan Zheng [view email]
[v1]
Wed, 14 Oct 2026 02:24:11 UTC (200 KB)
来源：arXiv cs.CL · arxiv.org