UDHRBench：面向大语言模型人权合规的基准测试

View PDF HTML (experimental)

Abstract:Large language models are increasingly deployed in contexts requiring adherence to ethical and legal norms, yet their alignment with internationally recognized human rights standards remains underexplored. We introduce UDHRBench, a large-scale benchmark for operationalising the evaluation of large language model compliance with the Universal Declaration of Human Rights. The dataset comprises 1,124 human rights confronting questions covering all 30 articles and 4,496 responses from four large language models (ChatGPT, Grok, Gemini, and Fanar), manually annotated by experts for alignment, with high inter-annotator agreement ($\kappa = 0.801$). Our analysis reveals systematic and non-trivial misalignments, showing that violations typically arise through seemingly rational reasoning: appeals to efficiency, risk management, or scientific neutrality, rather than explicit rejection of rights. We further propose an LLM-as-a-Judge framework for automated evaluation using zero-shot and few-shot prompting, achieving 89.8\% weighted F1-score in detecting human-rights misalignment, alongside a novel Alignment Leniency Score that quantifies how judge models resolve ambiguous cases. The dataset, annotations, and code will be released as open-source resources (this https URL).

Comments:	Alak Raad and Noreldin Morssy contributed equally to this work
Subjects:	Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
ACM classes:	I.2.7, K.4.1
Cite as:	arXiv:2610.15179 [cs.CL]
	(or arXiv:2610.15179v1 [cs.CL] for this version)
	https://doi.org/10.48550/arXiv.2610.15179 arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Serin Atiani [view email]
[v1] Sat, 10 Oct 2026 20:59:28 UTC (270 KB)