arXiv cs.CL· Avinash Holenarasipur Nagendra, Anand Kaniyaar Sriranga Prasad, Kousik Sarathy Sridharan, Mohan Raghavan·· 11 小时前AI 评分22
Dhārā Bench：面向印度知识系统的证据导向检索基准
Dh\={a}r\={a} Bench: An Evidence-Oriented Retrieval Benchmark for Indian Knowledge Systems
AI 导读
Dhārā Bench 是面向印度知识系统（IKS）的检索基准，包含 12 个类别的 325 个问题，语料为 15 个来源组的 60,121 个文本块，标注含 7,578 条问题-文本块相关性链接。
正文 · 原文
Abstract:Dhārā Bench is a retrieval benchmark for Indian Knowledge Systems (IKS), with 325 questions in 12 categories and 60,121 text chunks from 15 source groups. The categories were developed based on 4,500 questions submitted to the Dhārā application; the evaluation set combines corpus-aligned application queries with expert-contributed questions. The annotations comprise 7,578 question-chunk relevance links involving 6,847 distinct corpus chunks. For 34 composite and multi-hop questions, 401 conjugate groups specify alternative sets of passages that jointly support each question. Chunk identifiers and hierarchical references connect the judgments to their source texts. The paper describes corpus preparation, annotation, and validation, and reports a fixed BM25 baseline implemented with Pyserini with $k_1=0.9$ and $b=0.4$. Across all questions, BM25 achieves MRR@20 of 0.315, Recall@10 of 0.182, and nDCG@10 of 0.205. On the group-annotated subset, complete-evidence scores are CE_Recall@20 = 0.294 and CE_MRR@20 = 0.141. The category results expose particularly weak verse retrieval, while group scores reveal gaps between finding individual passages and completing their support. The corpus and evaluation code are available at this https URL.
| Comments: | 18 pages, 6 figures, 9 tables. The first two authors contributed equally. Code and data: this https URL |
| Subjects: | Information Retrieval (cs.IR); Computation and Language (cs.CL) |
| Cite as: | arXiv:2611.03196 [cs.IR] |
| (or arXiv:2611.03196v1 [cs.IR] for this version) | |
| https://doi.org/10.48550/arXiv.2611.03196 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Avinash Holenarasipur Nagendra [view email]
[v1]
Mon, 2 Nov 2026 14:23:39 UTC (104 KB)
来源：arXiv cs.CL · arxiv.org