arXiv cs.AI· Patrick Helm, Jan-Niklas Doerr, Joren Gijsbrechts, Stefan Minner·· 13 小时前AI 评分31
硬约束下的平滑梯度：通过可微分投影学习可行的库存策略
Hard Constraints, Smooth Gradients: Learning Feasible Inventory Policies via Differentiable Projection
AI 导读
研究者提出在策略网络内嵌入可微分优化模块：神经网络输出连续动作目标，二次规划将其投影到松弛可行集，再由对偶信息驱动的整数映射恢复整数性并保持可行性，使策略能像 MILP 一样灵活处理硬约束。
正文 · 原文
Abstract:Many operational problems are constrained sequential decision processes with combinatorial action spaces and interdependent constraints. Mixed-integer linear programs (MILPs) handle such constraints flexibly but scale poorly in stochastic environments. Deep reinforcement learning (DRL) promises scalable decision rules, but existing methods either penalize constraints rather than enforce them, or rely on feasibility mechanisms that break down once constraints interact. We bridge this gap by embedding a differentiable optimization module inside the policy: a neural network proposes continuous action targets, a quadratic program projects them onto the relaxed feasible set, and a dual-informed integer mapping restores integrality while preserving feasibility. Given a differentiable simulator, the policy trains end-to-end from sampled trajectories using pathwise gradients, while handling hard constraints with similar flexibility to MILPs. We show that our approach has bounded error relative to an exact integer projection and ensures the entire feasible action space is reachable. We apply the method to multi-echelon production-inventory planning under shared resource and material constraints. Our policy attains an average optimality gap of 1% on 3-item instances and consistently outperforms a rolling-horizon multi-stage stochastic program by at least 5.6% in 10-item networks. Compared to state-of-the-art echelon base-stock policies, savings reach 15.3% on 40-item instances. On an industry-scale case study with nonstationary demand from ASML, we reduce average cost by up to 1.8% relative to the best-known benchmark policy. The savings are largest where planning is hardest: in tightly capacitated systems with high demand variability. More broadly, we show that DRL can improve upon state-of-the-art baselines in sequential decision problems with hard constraints, which are widespread in practice.
| Subjects: | Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2608.02343 [cs.AI] |
| (or arXiv:2608.02343v2 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2608.02343 arXiv-issued DOI via DataCite |
Submission history
From: Patrick Helm [view email]
[v1]
Mon, 3 Aug 2026 14:57:21 UTC (109 KB)
[v2]
Tue, 27 Oct 2026 14:59:25 UTC (149 KB)
来源：arXiv cs.AI · arxiv.org