arXiv cs.CL· Hieu Tran, Hong Yu·· 13 小时前AI 评分34
强化学习究竟教会了 LLM 智能体什么？一项审计市场合谋执法机制的研究
Is Anyone Enforcing the Cartel? What Reinforcement Learning Teaches LLM Agents in Markets
AI 导读
研究提出一种审计方法，通过在市场的反事实副本中强制一名成员作弊，检验 LLM 定价智能体是否真正执行合谋。经 RL 训练的小型 LLM 只学会训练对手所奖励的行为：面对惩罚型对手学会被威慑，面对作弊型对手学会跟随降价，却几乎不会尝试能威慑对手的价格战。Q-learner 仅在定价场景中执行合谋，提示词驱动的 LLM 多靠忽略市场的固定规则达成类卡特尔结果，而结果指标无法区分这些情况。
正文 · 原文
Abstract:Evaluations of large language model (LLM) agents report that pricing agents collude at others' expense. Yet collusion is defined by enforcement: a cartel, in which competitors share monopoly profits, holds only if a member who cheats is punished. We introduce an audit that tests enforcement by forcing a member to cheat in counterfactual copies of a market, and use it to study what reinforcement learning (RL) teaches agents in pricing and trading. Small LLMs trained by RL learn what their training rivals reward, but only what they readily try. In self-play, they learn fixed rules ignoring their rivals; against a punishing rival, to be deterred; against a cheating rival, to match its cuts, never the costly price war that would deter it, which they rarely try. A war forced into training is learned, but mostly only where it was forced. Against their own copies, deterred and matching agents can earn the same cartel profits, but only matching ones can enforce them, and then thinly: as a by-product and only against large cuts. Q-learners enforce collusion only in pricing. Prompted LLMs reach cartel-like outcomes mostly through rules that ignore the market. Outcome measures cannot tell these cases apart; the audit can.
| Comments: | 45 pages, 13 figures, 20 tables |
| Subjects: | Computation and Language (cs.CL); Computer Science and Game Theory (cs.GT); Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.28787 [cs.CL] |
| (or arXiv:2610.28787v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.28787 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Hieu Tran [view email]
[v1]
Tue, 20 Oct 2026 18:18:10 UTC (378 KB)
来源：arXiv cs.CL · arxiv.org