arXiv cs.CL· Brianne Lee·· 12 小时前AI 评分36
17 岁学生写的答案被算在谁头上？六款前沿语言模型作者年龄误判的行为案例研究
Which Answer Did the Seventeen-Year-Old Write? A Behavioral Case Study of Author-Age Misattribution, Miscalibrated Confidence, and Non-Corrective Self-Explanation in Six Frontier Language Models
AI 导读
一项针对 Claude Fable 5、Gemini 3.1 Pro、GPT-5.6 Sol、Grok 4.5、GPT-4o (2024) 和 DeepSeek V4 六款模型的行为案例研究显示。
正文 · 原文
Abstract:Three students answered the same exam prompt by hand: "What is happiness? Are you happy?" Their ages, recorded at collection, are 17, 28, and 45. The 17-year-old's answer is hedged and formally phrased. Asked to match each answer to an age and gender label, most frontier language models confidently assign it to an adult. This paper reports a structured case study of that failure across six models: Claude Fable 5, Gemini 3.1 Pro, GPT-5.6 Sol, Grok 4.5, GPT-4o (2024), and DeepSeek V4. We ran 1,081 API sessions and scored 3,225 responses across six experimental conditions that manipulate framing, disclosure format, and audience-awareness instruction. Three of six models perform at or near chance (1/3) on the matching task while reporting 73-88% mean confidence. The youngest writer's answer is misattributed to an adult in up to 90% of trials. Removing the multiple-choice scaffold collapses accuracy further: three models never place within three years of the true age once open-ended estimation is required. Institutional framing ("a community-college biology final") significantly degrades the best-performing model's accuracy; a neutral label does not. A single audience-awareness instruction significantly improves one model (+21 points) and moves another to ceiling, with no detectable effect on the three weakest models. Stated confidence is uncorrelated or inversely correlated with correctness for five of six models. After ground truth is revealed, one model names its own stereotyping in 68 of 114 explanations, yet shows the lowest accuracy and highest confidence of any model under blind presentation: articulate retrospective diagnosis that does not track the quality of its judgments. This is a three-item, single-topic, single-language case study meant to isolate a mechanism, not to estimate a population bias rate. Data, code, and raw model outputs are publicly released.
| Comments: | 15 pages, 6 tables. Code and data: this https URL |
| Subjects: | Computation and Language (cs.CL) |
| ACM classes: | I.2.7 |
| Cite as: | arXiv:2610.31946 [cs.CL] |
| (or arXiv:2610.31946v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.31946 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Brianne Lee [view email]
[v1]
Thu, 22 Oct 2026 21:45:19 UTC (18 KB)
来源：arXiv cs.CL · arxiv.org