arXiv cs.AI· Bingnan Zhao, Harshala Gammulle, Tharindu Fernando·· 4 小时前AI 评分31
深度强化学习在 HVAC 监督控制中何时真正有效?四项直接在 FMU 上训练的 Modelica 小型案例研究
When does deep reinforcement learning pay off in HVAC supervisory control? Four small Modelica case studies trained directly on FMUs
AI 导读
一项基于布里斯班办公楼四项 Modelica 小型案例的研究发现,直接在导出的 FMU 上训练的 soft actor-critic 智能体在无储能的冷凝水与冷冻水重置案例中达到近最优参考,但相对调优规则控制器的优势仅 1.3% 和 0.18%。
正文 · 原文
Abstract:Deep reinforcement learning (DRL) is often proposed for the supervisory control of heating, ventilation and air-conditioning (HVAC) systems. The reported gains are, however, difficult to interpret, because the baselines are often weak and the attainable optimum is seldom known. This paper examines four small, reproducible case studies based on an office building in Brisbane, Australia: pre-cooling under a time-of-use tariff; chiller staging with condenser water reset; chilled water supply temperature reset with a wet cooling coil; and chilled water storage under time-of-use prices and a demand charge. The plants were modelled with the Modelica Buildings library, and in three of the cases a soft actor-critic agent was trained directly on the exported functional mock-up unit (FMU). Each agent was compared with tuned rule-based controllers and, where practicable, with a near-optimal reference from hourly exhaustive search or dynamic programming with perfect foresight.
In the two cases without storage the agent reached the near-optimal reference, but the margin above a well-chosen rule was small: 1.3% for condenser water reset and 0.18% for chilled water reset. In the latter case two common reset rules consumed more energy than a fixed set point. With a chilled water storage tank the margin was considerably larger. The agent outperformed both rules (-7.8%, compared with -3.9% and -6.3%) and adopted midday charging during the solar off-peak period, yet it obtained only about 45% of the saving achieved by the perfect-foresight schedule and did little to limit the demand peak. In the pre-cooling case a hand-tuned rule remained superior (-12.0% compared with -5.6%). These results suggest that studies of DRL for HVAC control should state the margin between the best rule and a near-optimal reference. Some practical difficulties encountered in training on co-simulation FMUs are also reported.
| Comments: | 16 pages, 4 figures, 2 tables. Code, models and trained agents: this https URL |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Systems and Control (eess.SY) |
| Cite as: | arXiv:2611.22144 [cs.LG] |
| (or arXiv:2611.22144v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2611.22144 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Bingnan Zhao [view email]
[v1]
Tue, 17 Nov 2026 03:29:06 UTC (569 KB)
来源:arXiv cs.AI · arxiv.org