arXiv cs.CL· Junhyeok Lee, Han Jang, Kyu Sung Choi·· 9 天前AI 评分22
MoE 语言模型中的路由重加权难以有效控制刻板印象
Limited Stereotype Control Through Routing Reweighting in MoE Language Models
AI 导读
研究提出诊断框架 FARE，结合人口统计路由画像、经验层选择与固定推理时重加权，在五种 MoE 架构上测试路由级刻板印象控制。在选定操作点上，CrowS-Pairs 偏好变化最多 1.3 个百分点，DeepSeekMoE 未选择任何干预，唯一名义显著变化（Qwen1.5，p = 0.015）未通过多重比较校正。
正文 · 原文
Abstract:Demographic prompts are routed differently from neutral prompts in Mixture-of-Experts (MoE) language models, motivating tests of routing-level stereotype control. We introduce Fairness-Aware Routing Equilibrium (FARE), a diagnostic framework combining demographic routing profiles, empirical layer selection, and fixed inference-time reweighting, and evaluate five MoE architectures in English. At the selected operating points, CrowS-Pairs preference changes by at most 1.3 percentage points; DeepSeekMoE selects no intervention. Paired 95% confidence intervals exclude decreases larger than 2.2 points on each intervened model, and the only nominally significant change (Qwen1.5, p = 0.015) does not survive multiple-comparison correction. OLMoE and Qwen3 nevertheless change nearly every top-k expert set. Noise controls, random and truncated synthetic profiles, and hard masking also move preference by at most 1.5 points. Four generation protocols on OLMoE, Mixtral, Qwen1.5, and Qwen3 show no consistent change in the measured toxicity, lexical, or reference-overlap metrics. Selection and evaluation items overlap, so these comparisons are not independent evaluations. The tested reweighting procedure offers limited stereotype control.
| Comments: | 15 pages, 4 figures, 12 tables |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2603.27141 [cs.CL] |
| (or arXiv:2603.27141v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2603.27141 arXiv-issued DOI via DataCite |
Submission history
From: Junhyeok Lee [view email]
[v1]
Sat, 28 Mar 2026 05:33:42 UTC (493 KB)
[v2]
Thu, 8 Oct 2026 07:58:19 UTC (213 KB)
来源：arXiv cs.CL · arxiv.org