arXiv cs.CY· Tunazzina Islam·· 9 小时前AI 评分24
当目标与评估器不一致:评估事后人口统计框架缓解的迁移性
When Objectives and Evaluators Disagree: Evaluating Transfer for Post-hoc Demographic Framing Mitigation
AI 导读
研究将 LLM 生成文本的事后改写缓解视为迁移问题,在 660 组性别对和 1,980 组年龄对比的气候信息上测试。始终针对 Persuasion Bias Index(PBI)较低消息的固定策略在不同轴和聚合方式下迁移不一致;自适应策略在两侧提出编辑但最多接受一个,在两个评估器家族的八项比较中有七项优于该规则,主要靠编辑更多消息对而非引导方向选择,且保留风险更高。
正文 · 原文
Abstract:Post-hoc rewriting is used to reduce demographic disparities in LLM-generated text when the source model is inaccessible. A rewrite can look successful merely because it improved its own target or satisfied its selector. We study this as a transfer problem on 660 gender pairs and 1,980 age contrasts of demographic-conditioned climate messages. A rewriter proposes edits, a selection model chooses among constraint-satisfying candidates, and an independent evaluator uninvolved in either rates pairwise framing disparity. A fixed policy that always targets the message with the lower Persuasion Bias Index (PBI) transfers inconsistently across axes and outcome aggregations. An adaptive policy that proposes edits on both sides but accepts at most one significantly improves on that rule, applied to the same candidates, in seven of eight comparisons across two evaluator families, mostly by editing more pairs, not by guided direction choice, at higher preservation risk. A policy editing both messages gives the largest point reductions on fewer pairs. These two policies reduce disparity across a preservation-threshold sweep; absolute ratings vary across evaluator families and prompts. Shrinking the PBI gap increases dimension-mean disparity under both evaluator families. Mitigation should be reported as transfer across objectives, evaluators and their prompts, and editing policies.
| Comments: | Accepted to EACL 2027 Main Conference |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Computers and Society (cs.CY); Machine Learning (cs.LG); Social and Information Networks (cs.SI) |
| Cite as: | arXiv:2611.25258 [cs.CL] |
| (or arXiv:2611.25258v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2611.25258 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Tunazzina Islam [view email]
[v1]
Wed, 18 Nov 2026 22:42:03 UTC (2,285 KB)
来源:arXiv cs.CY · arxiv.org