arXiv cs.CL· Naifan Zhang, Ruihan Sun, Jinwei Su, Hengjie Yang, Zhengyuan Pan, Zhaohan Chen, Xiaofan Zhang·· 13 小时前AI 评分15
MICA：面向长程情感支持对话的多粒度跨时间信用分配框架（论文已撤稿）
MICA: Multi-granularity Intertemporal Credit Assignment for Long-Horizon Emotional Support Dialogue
AI 导读
论文 MICA 提出一种无 critic 的强化学习框架，用于多轮情感支持对话的逐轮信用分配，通过共享势函数同时导出即时与延迟信用，无需匹配状态比较或 rollout 树。
正文 · 原文
This paper has been withdrawn by Jinwei Su
No PDF available, click to view other formats
Abstract:Reinforcement learning (RL) for large language models (LLMs) has shown strong performance in single-turn tasks, but extending it to multi-turn interaction remains challenging due to sparse rewards and poor per-turn credit assignment. In emotional support dialogues, responses shape future user states, so matched-state step-wise comparison is unavailable, while trajectory-level supervision is insufficient. We propose MICA (Multi-granularity Intertemporal Credit Assignment), a critic-free RL framework for multi-turn emotional support tasks. MICA derives both immediate and delayed credit from a shared potential function over the user's structured support state. Incremental Distance Reward measures the per-turn decrease in residual distance to the target state, while its Monte Carlo return captures delayed effects. After scope-specific normalization, the two signals form a mixed advantage for stable per-turn optimization without matched-state comparisons, rollout trees, or a learned critic. On EMPA, EQ-Bench, and EmoBench with Qwen2.5-7B-Instruct and Qwen3-8B/14B/32B, MICA consistently outperforms GRPO and REINFORCE++, achieving up to +42.5 on EMPA, while adding no rollout cost and remaining robust to reward judges. These results show that turn-aware credit assignment enables effective and practical multi-turn RL for interactive LLMs.
| Comments: | The authors have identified that additional experiments and corrections are required. To prevent the dissemination of potentially inaccurate results, we request the withdrawal of this paper. A revised version will be submitted once the additional work is complete |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2603.06194 [cs.CL] |
| (or arXiv:2603.06194v4 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2603.06194 arXiv-issued DOI via DataCite |
Submission history
From: Jinwei Su [view email]
[v1]
Fri, 6 Mar 2026 12:06:57 UTC (3,935 KB)
[v2]
Tue, 5 May 2026 15:53:05 UTC (6,532 KB)
[v3]
Wed, 29 Jul 2026 06:55:29 UTC (6,582 KB)
[v4]
Wed, 21 Oct 2026 08:26:01 UTC (1 KB) (withdrawn)
来源：arXiv cs.CL · arxiv.org