arXiv cs.CL· Boqin Yuan, Ziqi Wang·· 11 小时前AI 评分31
L-MARS：带智能体搜索与引用忠实度审计的法律多智能体系统
L-MARS: Legal Multi-Agent System with Agentic Search and Citation-Faithfulness Audit
AI 导读
L-MARS 是一个开源法律问答多智能体系统，通过智能体搜索与裁判驱动的证据核查，对每条原子主张逐条审计其引用来源。在 100 题 Bar Exam 分层审计中，检索对准确率提升有限，但多轮裁判循环将 strict citation F1 从 0.13（朴素 RAG）提升至 0.25，并将无引用率从 34% 降至 13%。
正文 · 原文
Abstract:Large language models are increasingly deployed for legal question answering, where evaluations typically focus on multiple-choice accuracy. This measure overlooks a common failure: whether the citation source attached to an answer exists and supports the rule the system attributes to it. We present L-MARS, an open multi-agent legal QA system with agentic search and judge-driven evidence checks, and audit it claim by claim against its cited source. Each atomic claim is labelled with a six-class taxonomy and scored with strict-ALCE under cross-provider judging, where the answerer and verifier come from different model families. On a stratified 100-question Bar Exam audit, retrieval barely moves accuracy, yet the multi-turn judge loop lifts strict citation F1 from 0.13 (naive RAG) to 0.25 and cuts the no-citation rate from 34% to 13%. We further introduce Faith-Search, a post-draft step that re-verifies and repairs unreachable citations; it drops the unreachable rate below 1% but does not improve F1 over the multi-turn loop, so we report it as a targeted reachability intervention rather than a faithfulness breakthrough. A 50-question LegalSearchQA case study confirms the picture: retrieve-then-draft pipelines saturate near 0.75 citation F1, while a single-agent web-search baseline collapses to 0.22 under external audit.
| Comments: | 14 pages, 2 figures, 4 tables. Accepted at the AI for Law Workshop at ICML 2026 (PMLR 356) |
| Subjects: | Artificial Intelligence (cs.AI); Computation and Language (cs.CL) |
| Cite as: | arXiv:2509.00761 [cs.AI] |
| (or arXiv:2509.00761v5 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2509.00761 arXiv-issued DOI via DataCite |
Submission history
From: Boqin Yuan [view email]
[v1]
Sun, 31 Aug 2025 09:23:26 UTC (912 KB)
[v2]
Wed, 3 Sep 2025 00:57:14 UTC (912 KB)
[v3]
Mon, 30 Mar 2026 02:42:59 UTC (857 KB)
[v4]
Wed, 15 Jul 2026 19:38:53 UTC (1,248 KB)
[v5]
Thu, 5 Nov 2026 23:11:59 UTC (271 KB)
来源：arXiv cs.CL · arxiv.org