AfriGuard：面向非洲语言的安全与稳健拒绝行为数据集
AfriGuard: Safety and Robust Refusal For African Languages
研究者推出 AfriGuard，一个覆盖 10 种非洲语言和英语、文化根基化的多语言数据集，用于研究有害意图、安全类别理解与拒绝行为。实验显示 guardrail 模型在安全意图与类别分类上不如开源和闭源 LLM，模型对恰当安全行为的理解有限，F1 低于 60%。基于该数据集开发的 guardrail 模型性能可与大至 30 倍的模型竞争，新拒绝检测器平均性能提升 43.9%。
Authors:Israel Abebe Azime, Jesujoba O. Alabi, Iffat Maab, Alexander D. Cai, Tadesse Destaw Belay, Atnafu Lambebo Tonja, Jonas Kgomo, Folasade Alabi, Salomey Osei, Saminu Muhammad Aliyu, Nkechinyere Faith Aguobi, Bontu Fufa Balcha, Blessing Sibanda, Davis David, Mouhamadane Mboup, Daud Abolade, Neo Putini, Louis Powell, Kiant'e Brantley, Philipp Slusallek, Dietrich Klakow, Julia Kreutzer
Abstract:Large language models (LLMs) designed to interact with users are typically trained to respond appropriately to harmful or toxic content. However, ensuring appropriate responses becomes more challenging in diverse linguistic and cultural contexts, where safety risks may be expressed or interpreted differently. Building safe AI systems therefore requires detecting safety risks, understanding their nature, and responding appropriately, while accounting for linguistic and cultural variations that add complexity to each stage. In this work, we introduce AfriGuard, a culturally grounded multilingual dataset for studying harmful intent, safety-category understanding, and refusal behavior across 10 African languages and English. Our findings reveal that guardrail models underperform both open- and closed-source LLMs on safety-intent and safety-category classification, while models show limited understanding of appropriate safety behavior, scoring below 60% F1. We further find that models often fail to respond appropriately to unsafe prompts not because they explicitly refuse, but because they produce linguistically inadequate responses that result in incidental non-assistance rather than proper refusals. Finally, using AfriGuard, we develop efficient guardrail models that achieve performance competitive with models up to 30x larger, introduce refusal detectors that improve average performance by 43.9% over existing detectors, and demonstrate that supervised fine-tuning can substantially improve overall safety performance.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.26474 [cs.CL] |
| (or arXiv:2610.26474v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.26474 arXiv-issued DOI via DataCite |
Submission history
From: Israel Abebe Azime [view email]
[v1]
Mon, 19 Oct 2026 14:21:55 UTC (1,956 KB)
[v2]
Thu, 22 Oct 2026 18:57:19 UTC (1,956 KB)
来源：arXiv cs.CL · arxiv.org