arXiv cs.CL· Jihao Zhao, Shuaishuai Zu, Zhiyuan Ji, Chunlai Zhou, Biao Qin·· 2 天前AI 评分27
Writer-R1：面向开放式写作强化学习的动态评分标准挖掘
Writer-R1: Dynamic Rubric Mining for Reinforcement Learning of Open-Ended Writing
AI 导读
Writer-R1 是一套围绕归纳式评分标准生成（IRG）构建的写作训练流程，将评分标准同时用作生成引导与强化学习奖励依据，并训练出 4B 的 RubricGen-4B 为新请求自动提供评分标准。它在 4B 和 8B 基座上将 WritingBench 分数分别提升 0.182 和 0.158，HelloBench 上也有增益，消融实验显示 IRG 优于直接生成评分标准。
正文 · 原文
Abstract:Training models for open-ended writing requires diverse prompts and sufficiently precise specifications of response quality. Constructing such specifications at scale entails recovering task requirements, consolidating overlapping criteria, and defining observable differences between performance levels. We introduce Writer-R1, a writing training pipeline built around Inductive Rubric Generation (IRG), which integrates these operations into an automated rubric construction procedure. The pipeline expands an open writing corpus and uses the resulting prompt-specific rubrics throughout supervised training and reinforcement learning (RL). Each rubric is concatenated with the user request and also defines the reward criteria, making the requirements used for assessment available to the writer before generation. A compact model, RubricGen-4B, learns to supply rubrics for new requests. Writer-R1 improves WritingBench scores by 0.182 and 0.158 over the 4B and 8B backbones, respectively, with gains also on HelloBench. Training ablations favor IRG over direct rubric generation, and further analyses support rubric transfer and the robustness of writer improvements across evaluators. These results support using inductively mined rubrics as both generation guidance and reward specifications for open-ended writing.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2603.15061 [cs.CL] |
| (or arXiv:2603.15061v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2603.15061 arXiv-issued DOI via DataCite |
Submission history
From: Jihao Zhao [view email]
[v1]
Mon, 16 Mar 2026 10:18:20 UTC (9,718 KB)
[v2]
Thu, 15 Oct 2026 02:36:00 UTC (13,203 KB)
来源：arXiv cs.CL · arxiv.org