GRAP：面向 LLM 性别偏见分析的性别评分标准感知探测框架

GRAP: Gender Rubric-Aware Probing for Analyzing Gender Bias in LLMs

AI 导读

GRAP 是一个性别评分标准感知探测框架，用轻量级 logistic 探针从 10 个评分维度的开放式指令跟随回答中识别最具区分度的偏见维度。它针对现有探测方法依赖固定格式任务、且仅比较原始组级评分而忽略各维度方差的问题，以任务无关方式运行。该工作已被 ECML-PKDD 2026 Workshop on Bias and Fairness in AI 接收。

正文 · 原文

View PDF HTML (experimental)

Abstract:As LLMs increasingly mediate everyday interactions, subtle gender biases in open-ended responses pose growing societal risks. Existing probing methods have two key limitations: they rely on fixed-format tasks, such as pronoun resolution or occupation association, that alignment-tuned models can easily handle; and they compare raw group-level rubric scores, ignoring per-dimension variance and obscuring which bias dimensions reliably distinguish gender groups. We introduce GRAP (Gender Rubric-Aware Probing), a framework that uses a lightweight logistic probe to identify the most discriminative bias dimensions from responses scored across 10 rubric dimensions, operating task-agnostically on open-ended instruction following.

Comments:	3 pages. Accepted at the ECML-PKDD 2026 Workshop on Bias and Fairness in AI
Subjects:	Artificial Intelligence (cs.AI)
Cite as:	arXiv:2611.13024 [cs.AI]
	(or arXiv:2611.13024v1 [cs.AI] for this version)
	https://doi.org/10.48550/arXiv.2611.13024 arXiv-issued DOI via DataCite

Submission history

From: Seona Seo [view email]
[v1] Mon, 9 Nov 2026 08:09:28 UTC (3,364 KB)

来源：arXiv cs.AI · arxiv.org