arXiv cs.CL· Ilia Karpov, Maksim Grigorenko, Vadim Ayuyev·· 11 小时前AI 评分34
MafiaScope：用社交推理游戏 Mafia 测量多智能体推理能力
MafiaScope: Inspecting Multi-Agent Reasoning in Social Deduction Games
AI 导读
MafiaScope 是一个开源测试平台，将社交推理游戏 Mafia 转化为多智能体推理的测量工具，可在每次公开发言后让存活智能体私下回答结构化问题，且问答不进入游戏历史、不干扰后续对局。角色判断按引擎已知角色计分，二阶预测与其他智能体的报告比对，可视化器支持冒充模式、时间线准确率与校准面板及反事实回放。实验覆盖多个模型、语言和探测配置，演示包含校准、探测可重复性与替换发言回放等检查。
正文 · 原文
Abstract:An LLM agent's public behaviour reveals little about its social reasoning: an agent that votes correctly may be guessing, and its public claims may differ from privately elicited assessments. We present MafiaScope, an open testbed that turns the social deduction game Mafia into a measurement instrument for multi-agent reasoning. After each public utterance, living agents privately answer configurable structured questions on copies of their available conversations. Probing is non-invasive with respect to the acting context: neither the questions nor the answers enter the histories used to continue the game. Role assessments are scored against roles known to the engine; second-order predictions are compared with other agents' reports. An interactive visualizer renders the assessment trajectories: impersonate mode focuses on one agent's information and reports, panels chart timeline-aligned accuracy and calibration, and counterfactual replay branches from saved states. An offline heuristic links shifts in collective suspicion to episodes for inspection. Experiments span multiple models, languages, and probing configurations; the demonstration presents episode inspection, calibration, probe-repeatability and probe/no-probe checks, and utterance-replacement replay. Code: this https URL Demo: this https URL Video: this https URL
| Subjects: | Computation and Language (cs.CL); Multiagent Systems (cs.MA); Social and Information Networks (cs.SI) |
| Cite as: | arXiv:2607.10645 [cs.CL] |
| (or arXiv:2607.10645v3 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2607.10645 arXiv-issued DOI via DataCite |
Submission history
From: Ilia Karpov [view email]
[v1]
Sun, 12 Jul 2026 08:20:52 UTC (1,372 KB)
[v2]
Sat, 18 Jul 2026 17:54:54 UTC (1,131 KB)
[v3]
Fri, 30 Oct 2026 20:23:31 UTC (750 KB)
来源：arXiv cs.CL · arxiv.org