arXiv cs.AI· Wenbin Zhang, ShangZhan Li, Bingxiang He, Ye He, Yuxin Zhou, Muzhen Cai, Qi Shi, Wanxiang Che·· 12 小时前AI 评分32
LexPilot：面向中文民事案件咨询的法律 LLM 主动交互能力基准测试
LexPilot: Benchmarking the Proactive Interaction Ability of Legal LLMs with Laypeople in Chinese Civil Case Consultation
AI 导读
研究者提出 LexPilot 基准，用 432 份真实中国民事判决构建 36 类案由的完整与信息缺失咨询对话对，并附专家审核与法院判决，用于衡量法律 LLM 的主动交互能力。实验显示，当前 LLM 在信息缺失咨询中主动追问能力不足，下游判决预测较完整咨询场景平均下降 23.84%。
正文 · 原文
Abstract:Large Language Models (LLMs) have greatly assisted legal professionals. However, our empirical studies reveal that, when faced with underspecified consultations from non-experts, current LLMs often produce irresponsible responses with limited utility despite lacking sufficient case details for definitive conclusions. To better assist laypeople, legal LLMs must identify focal points of dispute and proactively elicit crucial context from users. Unfortunately, existing data resources often focus on interaction and response quality, offering limited insight into how proactive interaction affects LLMs' downstream claim-level judgment prediction in actual civil cases. To address this gap, we introduce \textbf{LexPilot}, a benchmark linking proactive legal interaction to claim-level verdict prediction using authentic Chinese civil judgments. Constructed from 432 real-world judgments across 36 common causes of action, LexPilot comprises expert-reviewed pairs of complete and underspecified consultation histories with corresponding court verdicts. Experiments reveal that current LLMs struggle with proactive legal interactions in underspecified consultations, with downstream judgment prediction substantially underperforming that in the fully specified consultation setup (-23.84% on average). Further analysis identifies critical bottlenecks hindering these capabilities. Our work highlights the urgent need for more user-centered legal AI to better serve the public.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.23141 [cs.AI] |
| (or arXiv:2610.23141v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.23141 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Wenbin Zhang [view email]
[v1]
Fri, 16 Oct 2026 08:38:20 UTC (964 KB)
来源：arXiv cs.AI · arxiv.org