arXiv cs.CY· Colin Priest·· 11 小时前AI 评分42
相同事实，不同救济：LLM 与 System One 决策中的语言画像与任意性
Same Facts, Different Remedy: Linguistic Profiling and Arbitrariness in LLM and System One Decisions
AI 导读
研究审计了 GPT-4o-mini、GPT-5-mini 和概率模型 Jev 三个消费者投诉救济决策系统，用保留全部事实的社会语言学改写测试其公平性。群体层面，年长、受过大学教育和高 SES 写作风格获得略更宽松的救济，所有金钱救济差距在 0.5 个百分点以内；个体层面，同一投诉的两种风格版本有 2.6-6.3% 的概率得到不同金钱裁决。
正文 · 原文
Abstract:Large language models are increasingly used to resolve consumer complaints, and the way complainants write varies with age, education and socioeconomic status (SES). We audit three remedy-decision systems using sociolinguistic rewrites of US consumer financial complaints that are instructed to preserve every fact: two generative LLMs that return a single label (GPT-4o-mini and GPT-5-mini) and a probabilistic "System One" model that returns a distribution over remedies (Jev). Using pre-registered hypotheses tested once on held-out complaints, we separate two phenomena that single-label audits conflate. At the group scale, older, college-educated and higher-SES styles receive slightly more generous remedies; these effects replicate but are small, with every monetary-remedy gap within 0.5 percentage points. At the individual scale, two style versions of the same complaint receive different monetary decisions 2.6-6.3% of the time. In a variance-component analysis of the probabilistic model, persona components shared across the two rewrite models account for about 1.6% of within-complaint variation. The persona gaps are associated mainly with delivery (vocabulary, grammar, politeness) rather than content. Positive controls show that all three systems respond 49-387 times more strongly to remedy-relevant facts than to style. Rewriting every complaint into one standard style reduced GPT-4o-mini's education and SES gaps by 94% and 87% while retaining fact sensitivity, but reversed two of the probabilistic model's gaps and did not make individual decisions more stable. Independent LLM auditors flagged material fact changes in 3.3% and 11.7% of rewrites, and instability persisted among unflagged rewrites. Fairness audits of LLM decision systems should report group disparities and individual instability as separate outcomes.
| Comments: | 29 pages, 7 figures. Code, prompts and pre-registrations: this https URL |
| Subjects: | Computers and Society (cs.CY); Artificial Intelligence (cs.AI); Computation and Language (cs.CL) |
| Cite as: | arXiv:2611.08995 [cs.CY] |
| (or arXiv:2611.08995v1 [cs.CY] for this version) | |
| https://doi.org/10.48550/arXiv.2611.08995 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Colin Priest [view email]
[v1]
Fri, 6 Nov 2026 05:10:09 UTC (83 KB)
来源：arXiv cs.CY · arxiv.org