arXiv cs.CY· Rusali Saha, Nitya Khurana, Ruihong Huang, Kuan-Hao Huang·· 3 天前AI 评分31
ChronoCulture：评估大语言模型跨时间文化意识的新基准
ChronoCulture: Evaluating Cultural Awareness Across Time in Large Language Models
AI 导读
研究者推出 ChronoCulture 基准，包含覆盖 72 个国家、基于纵向调查数据构建的 10,360 个时间配对，每个配对含同一国家同一调查问题在新旧两个年份的完整人口回答分布。在六个 LLM 上测试发现，模型常能预测某国某一时点的态度，却无法可靠捕捉态度随时间的变化，其不同年份预测往往彼此接近；在提示词中指定目标年份也几乎不能提升与调查回答的对齐度。
正文 · 原文
Abstract:Cultural attitudes and social norms evolve over time, yet most cultural benchmarks for large language models (LLMs) treat culture as a static attribute of a country or region. We introduce ChronoCulture, a benchmark of 10,360 temporal pairs built from longitudinal survey data across 72 countries. Each pair contains the same survey question asked in the same country in an older and a newer year, with full population response distributions as ground truth. Across six LLMs, we find that models can often predict a country's attitudes at one point in time but do not reliably capture how those attitudes change. Their predictions for different years tend to stay close together, failing to capture much of the observed change. Specifying the intended year in the prompt also offers little overall improvement in alignment with survey responses. These findings suggest that models often present a largely static picture of cultural attitudes, highlighting the need to evaluate cultural awareness across time.
| Comments: | 28 pages (8 pages main text). Code: this https URL |
| Subjects: | Computation and Language (cs.CL); Computers and Society (cs.CY) |
| Cite as: | arXiv:2610.20472 [cs.CL] |
| (or arXiv:2610.20472v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.20472 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Rusali Saha [view email]
[v1]
Wed, 14 Oct 2026 16:50:35 UTC (8,554 KB)
来源：arXiv cs.CY · arxiv.org