arXiv cs.CL· Kuniko Paxton, Ifeoluwa Wuraola, Dhavalkumar Thakker·· 13 小时前AI 评分31
模型规模与量化如何影响问答中的社会偏见与拒答行为
Effects of Model Scale and Quantization on Social Bias and Abstention in Question Answering
AI 导读
研究基于 BBQ 基准测试 Qwen3、Gemma-3 和 SmolLM2 的多个规模与量化配置，发现更大模型问答准确率更高，但偏见无简单缩放趋势，量化影响因配置而异。26 个偏见回答更少的配置中有 20 个同时出现过度拒答增加，说明观测到的低偏见部分源于更多拒答。公平性评估需联合考虑偏见回答、拒答、提示词表述与答案选项设计。
正文 · 原文
Abstract:The deployment of small and quantized language models in resource-constrained settings raises an important fairness question: when a compressed model produces fewer biased responses, does this reflect fairer behavior or greater abstention? We study this problem in Question Answering (QA) using the BBQ benchmark across multiple model sizes and quantization settings from Qwen3, Gemma-3, and SmolLM2. We also examine whether the observed trends remain stable across alternative prompt formulations and when the ``Unknown'' option is removed in a forced-choice setting. Our results show that larger models achieved higher QA accuracy, but bias showed no simple scaling trend and quantization effects varied across configurations. Notably, 20 of the 26 configurations with fewer biased responses also show increased over-abstention, indicating that lower observed bias can partly reflect greater abstention. These findings show that lower observed bias can partly reflect greater abstention, and that fairness evaluation should consider biased responses, abstention, prompt formulation, and answer-option design jointly.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.29657 [cs.CL] |
| (or arXiv:2610.29657v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.29657 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Kuniko Paxton [view email]
[v1]
Wed, 21 Oct 2026 12:38:04 UTC (2,964 KB)
来源：arXiv cs.CL · arxiv.org