跳到正文
arXiv cs.AI· Marzieh Karimi, Mehran Safayani, Alireza Basiri·· 4 小时前AI 评分22

TUSI-Bench:面向波斯语法律的引文溯源图基准

TUSI: A Citation-Grounded Graph-Based Benchmark

AI 导读

研究者提出 TUSI-Bench,一个面向波斯语法律 RAG 系统的结构化基准,基于含 7,261 条法律条文的多层法律知识框架构建。该基准包含需显式法律引文的开放式问答集 TUSI-QA(1,999 题)和源自伊朗律师考试的四选一选择题集 TUSI-Bar(约 500 题),并配套构建法律知识图谱。实验显示引入图谱关系可使检索召回率较标准基线最高提升 8 个百分点。

正文 · 原文

View PDF HTML (experimental)

Abstract:Evaluating retrieval-augmented generation (RAG) systems in Persian law remains challenging because benchmarks lack realism, grounding, and coverage of legal reasoning. Existing resources are either synthetically generated from statutory texts, lacking natural information needs, or derived from informal legal question--answer sources without explicit linkage to authoritative legal documents, making them unsuitable for rigorous evaluation of grounded legal reasoning. To address this gap, we introduce TUSI-Bench, a structured Persian legal benchmark designed for the evaluation of RAG-based systems. TUSI-Bench is built upon a multi-layered legal knowledge framework containing 7,261 legal articles and integrating codified statutes, legal commentaries, doctrinal writings, and Supreme Court unification rulings. We also construct a legal knowledge graph linking articles and concepts for structure-aware retrieval. Based on this foundation, TUSI-Bench provides two complementary evaluation tasks: (i) TUSI-QA, an open-ended question answering benchmark containing 1,999 questions that requires grounded responses with explicit legal citations, and (ii) TUSI-Bar, a multiple-choice benchmark containing approximately 500 four-option questions derived from Iranian Bar examination questions. Furthermore, we propose a graph-based retrieval baseline that leverages relational structure within the legal knowledge graph. Experimental results demonstrate that incorporating graph-based relations improves retrieval recall by up to 8 percentage points compared to standard baselines, highlighting the importance of structured legal representations for RAG systems. The proposed benchmark provides a standardized resource for evaluating grounded legal reasoning in Persian and enables future research on structured legal intelligence.
Comments: 28 pages, 11 figures, including appendix
Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
Cite as: arXiv:2611.21823 [cs.CL]
  (or arXiv:2611.21823v1 [cs.CL] for this version)
  https://doi.org/10.48550/arXiv.2611.21823

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Marzieh Karimi [view email]
[v1] Mon, 16 Nov 2026 19:38:41 UTC (1,145 KB)

来源:arXiv cs.AI · arxiv.org