arXiv cs.CL· Julia Witte Zimmerman, Calla G. Beauregard, Tabia Tanzin Prama, Parisa Suchdev, Kathryn Cramer, Elisabeth Kollrack·· 14 小时前AI 评分38
LLMs 并非随机鹦鹉：conlang-like 任务证明其具备意义中介的抽象能力
LLMs are not stochastic parrots: Evidence for meaning-mediated abstraction from conlang-like tasks
AI 导读
研究用 conlang-like 任务测试多个 LLM：仅给出虚构语言的自然语言描述、不提供示例输出，且这些描述刻意违背训练数据中的表层统计模式。模型在三个互补任务族中系统性地朝意义预测方向作答，能区分“提示中出现”与“被指示使用”，并有时精确匹配复杂翻译答案。结果表明 LLM 存在意义中介的抽象，反驳强版随机鹦鹉假说，但生成仍受表层合理性强烈约束。
正文 · 原文
Abstract:The strong version of the stochastic parrot argument claims that, although large language models (LLMs) may exceed rote regurgitation, they cannot move beyond statistical pattern matching into abstraction or reasoning, remaining ontologically near the lower bound of pattern reuse despite producing alluringly fluent text. We test this hypothesis using conlang-like tasks. Several LLMs are given only natural-language descriptions of fictional languages that subvert prominent superficial patterns in training data by combining statistically uncommon and unattested features. Crucially, no example outputs are given. We argue that if the models exhibit rule-following behaviour, they cannot be relying solely on superficial statistical patterns; such patterns often work against the correct output. Instead, successful performance requires representations of the constraints specified in the prompt. Across three complementary task families, models systematically move in the meaning-predicted direction: they distinguish prompt exposure from instructed use, alter semantic relationships in response to novel constraints, and sometimes produce exact matches to complex translation answer keys. Although performance varies across the spectrum of models used, these results provide evidence for meaning-mediated abstraction in LLMs and refute the strong stochastic parrot hypothesis. Our work shows that, under appropriate architectural and contextual constraints, statistical learning can produce meaning-mediated abstractions, although generation remains strongly constrained by superficial plausibility. We discuss implications for model development and for understanding how increasingly abstract representations may emerge from plausible-text-generation objectives.
| Comments: | Presented as a poster at ISC Summer School 2026; related work presented at IC2S2 2026; a synthesis of this project and related work presented at SLSA/ 4S 2026 |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2609.34187 [cs.CL] |
| (or arXiv:2609.34187v4 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2609.34187 arXiv-issued DOI via DataCite |
Submission history
From: Julia Zimmerman [view email]
[v1]
Mon, 28 Sep 2026 03:03:12 UTC (11,363 KB)
[v2]
Tue, 29 Sep 2026 15:46:02 UTC (11,365 KB)
[v3]
Sun, 4 Oct 2026 22:55:09 UTC (11,367 KB)
[v4]
Sun, 25 Oct 2026 16:56:53 UTC (11,369 KB)
来源：arXiv cs.CL · arxiv.org