IndicDBBench：面向印度语言的多语言 Text-to-SQL 基准

View PDF HTML (experimental)

Abstract:While Large Language Models (LLMs) have significantly advanced Text-to-SQL performance, existing benchmarks largely focus on Western contexts leaving a gap in real-world, non-Western applications. We introduce IndicDBBench, a comprehensive multilingual Text-to-SQL benchmark designed to evaluate cross-lingual semantic parsing across diverse Indic language families. The underlying schemas are derived from real-world open-data platforms (NDAP and IDP), ensuring realistic structural complexity. IndicDBBench comprises 20 databases spanning 237 tables. To construct complex relational structures from denormalized government data, we employ a three-agent pipeline (Architect, Auditor, Refiner), achieving high relational density (11.85 tables per database) and join depths of up to six. We generate 15,612 tasks spanning seven languages: English and six Indic languages, ensuring that queries are grounded in real database values and require multi-table reasoning. We evaluate state-of-the-art models, including DeepSeek V3.2, MiniMax M2.7, Llama 3.3, and Qwen3, across seven linguistic variants. Our results reveal a 9.0-point drop in execution accuracy from English to Indic languages, exposing an "Indic Gap" that, for the two models tested, persists even when gold tables and literal values are supplied, since this grounding raises accuracy in every language by similar amounts. The dataset and code are publicly available at this https URL

Subjects:	Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Databases (cs.DB)
Cite as:	arXiv:2604.13686 [cs.CL]
	(or arXiv:2604.13686v2 [cs.CL] for this version)
	https://doi.org/10.48550/arXiv.2604.13686 arXiv-issued DOI via DataCite

Submission history

From: Roshan Karanth [view email]
[v1] Wed, 15 Apr 2026 10:07:37 UTC (10,912 KB)
[v2] Sun, 1 Nov 2026 14:09:37 UTC (6,320 KB)