arXiv cs.CL· Md Rayhanul Masud, S M Maksudul Alam, Mohammad Jaminur Islam·· 3 天前AI 评分22
BANGLAPLAN：孟加拉语约束规划与决策基准
BANGLAPLAN: A Benchmark for Constraint-Based Planning and Decision-Making in Bangla
AI 导读
研究者发布 BANGLAPLAN，一个覆盖十类任务、共 1,000 道题的孟加拉语约束规划与决策基准，含可行与不可行问题及四种语言语域。评测 11 个开源与闭源模型配置显示差距悬殊：GPT-6.1 Sol 准确率 98.7%，Qwen3.5-27B 开启 thinking 达 80.3%，而直接作答配置最高仅 30.9%，11 个配置中有 7 个低于“全部不可行”基线。
正文 · 原文
Abstract:Large language models (LLMs) increasingly help users arrange appointments, plan journeys, and allocate limited resources. These tasks require models to satisfy constraints on time, cost, capacity, and order, or determine that no feasible solution exists. Bangla is the world's seventh most widely spoken language, and existing benchmarks cover many language and reasoning tasks. However, benchmarks for evaluating planning and decision-making in Bangla under explicit constraints remain lacking. We introduce BANGLAPLAN, a benchmark of 1,000 problems across ten task families grounded in everyday life and institutions in Bangladesh. It includes feasible and infeasible problems in four language registers: everyday, formal, colloquial, and Bangla--English mixed. Answers cover eight structured types, including schedules, routes, allocations, and ordered steps, in a common JSON format. We construct the benchmark through language-model-assisted drafting and independent human validation. We evaluate 11 open-weight and proprietary model configurations using exact match, separately reporting feasible-case accuracy, infeasibility detection, and format errors. We find a large performance gap: GPT-6.1 Sol achieves 98.7% accuracy and Qwen3.5-27B with thinking reaches 80.3%, while direct-answer configurations score at most 30.9%. Seven of eleven configurations perform below the always-``infeasible'' baseline. Our findings also suggest that harder problems primarily reduce plan-construction accuracy, while infeasibility detection remains more robust. BANGLAPLAN provides a reproducible basis for evaluating planning and decision-making in Bangla. BANGLAPLAN is available at: this https URL.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.19704 [cs.CL] |
| (or arXiv:2610.19704v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.19704 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Md Rayhanul Masud [view email]
[v1]
Wed, 14 Oct 2026 07:32:32 UTC (225 KB)
来源：arXiv cs.CL · arxiv.org