On the use of conditional TimeGAN to enhance the robustness of a reinforcement learning agent in the building domain.

This paper develops an end-to-end data-driven pipeline to improve the out-of-sample performance of a Reinforcement Learning (RL) agent operating in the domain of building energy management. The approach can benefit researchers and practitioners that are confronted with the challenge of training robust control architectures when only few historical data are available to them. Under these circumstances, in fact, the RL agent is generally unable to respond robustly to unseen (possible, rare) events. To tackle this issue, we propose a data-driven procedure composed of two steps: (i) we develop a novel Generative Adversarial Network (GAN) architecture to create synthetic time series profiles of building performance; (ii) we infuse these artificial profiles into the original training dataset. The procedure is found to increase the robustness of the RL agent to rare events, without compromising the performance during "standard" operations. Extended simulations conducted on the CityLearn OpenAI Gym environement show that the GAN-enhanced RL agent's response displays better performance metrics with respect to a rule-based controller, with results generally improving with the data-enhancement process.