PLVR：用可验证奖励把专家轨迹蒸馏为程序

View PDF HTML (experimental)

Abstract:Small language models struggle with tool-use tasks that require following multi-step procedures. Whether these failures reflect limited capability or missing procedural knowledge remains unclear. We introduce Program Learning with Verifiable Rewards (PLVR), which distils expert traces into reusable programs without updating model weights. Frontier models turn recurring operations in the traces into typed primitives and rules, and search assembles them using reference actions and training-task checks. A small frozen model executes the primitives, with deterministic verifiers checking their outputs. The resulting program guides a student's reasoning or checks its proposed actions. We evaluate the same program across students on $\tau^2$-Bench Retail and Telecom and LiveMCPBench, using synthetic training tasks and unseen benchmark test tasks. On Telecom, PLVR raises an 8B model's success rate from 43.0% to 91.2%, while larger students reach up to 94.4%, compared with 57.9% for the best baseline. On Retail and LiveMCPBench, PLVR is competitive with the best baselines, although tool search remains more dependent on the student model. These results support program learning as a way to transfer expert procedures across frozen models, trading additional inference computation for more reliable tool use.

Subjects:	Artificial Intelligence (cs.AI); Symbolic Computation (cs.SC)
Cite as:	arXiv:2610.19148 [cs.AI]
	(or arXiv:2610.19148v3 [cs.AI] for this version)
	https://doi.org/10.48550/arXiv.2610.19148 arXiv-issued DOI via DataCite

Submission history

From: Asad Aali [view email]
[v1] Tue, 13 Oct 2026 20:12:54 UTC (61 KB)
[v2] Fri, 16 Oct 2026 18:02:03 UTC (62 KB)
[v3] Thu, 12 Nov 2026 00:38:11 UTC (58 KB)