arXiv cs.CY· Antony Dalmiere (LAAS-TRUST), Pascal Marchand (LAAS-TRUST, INSA Toulouse), Guillaume Auriol (LAAS-TRUST, INSA Toulouse), Vincent Nicomette (LAAS-TSF, LAAS)·· 10 天前AI 评分27
连续训练阶段与大语言模型说服力：错位、监督微调与偏好优化的影响
Successive Training Stages and Large Language Model Persuasion: Effects of Misalignment, Supervised Fine-Tuning, and Preference Optimization
AI 导读
一项研究考察了三个连续训练阶段对 LLM 说服力的影响：在阴谋论数据上做 SFT 造成错位、在论证数据上追加说服性 SFT、以及 IPO 偏好优化。835 名参与者被随机分入五组，结果显示说服性 SFT 带来的态度改变大于单纯的阴谋论训练（d = 0.30），而 IPO 没有额外收益（d = 0.03），GPT-4 与中性文本无差异（d = -0.01）。
正文 · 原文
Abstract:Large language models (LLMs) can be tuned to influence human attitudes, yet the respective contributions of successive post-training stages remain un-clear. This study examines how three successive training stages affect LLM persuasiveness: (1) misalignment through supervised fine-tuning (SFT) on conspiracy data, (2) additional persuasive SFT on argumentative data, and (3) Identity Preference Optimization (IPO), a preference-optimization method. A total of 835 participants recruited on Prolific were randomly assigned to five between-subject conditions (neutral text, conspiracy-trained model, persuasion-trained model, preference-optimized model, and GPT-4) and were exposed to texts on 10 divisive political issues, personalized from their individual profiles in all model conditions. Attitude change was measured as the difference between pre- and post-exposure positions on continuous Likert scales and analyzed with an analysis of covariance (ANCOVA). A significant condition x baseline-attitude interaction, F (4, 825) = 5.33, p < .001, indicated that training effects depended on participants' initial attitudes. Persuasive SFT produced greater attitude change than conspiracy training alone, d = 0.30, whereas IPO provided no additional benefit, d = 0.03, and GPT-4 did not differ from neutral text, d = --0.01. These results show that targeted supervised training on persuasive data increases LLM persuasiveness, whereas preference optimization yields no significant gains beyond it.
| Subjects: | Artificial Intelligence (cs.AI); Computers and Society (cs.CY) |
| Cite as: | arXiv:2610.09964 [cs.AI] |
| (or arXiv:2610.09964v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.09964 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Antony DALMIERE [view email] [via CCSD proxy]
[v1]
Wed, 7 Oct 2026 12:36:18 UTC (177 KB)
来源：arXiv cs.CY · arxiv.org