An integrated energy management system using double deep Q-learning and energy storage equipment to reduce energy cost in manufacturing under real-time pricing condition: A case study of scale-model factory

Reducing energy costs is an emerging aspect in the research on the economic and environmental dimensions of manufacturing systems. The share of electricity cost accounts for approximately 60 % of the total energy cost of a manufacturing system, whereas the share of oil, coal, and gas accounts for the remaining 40 %. The electricity cost is dependent on the electricity price and usage. In terms of the electricity price, one of the pricing strategies widely used in the USA and Europe is called real-time pricing (RTP), which is characterised by hourly price changes. Compared to other pricing strategies, RTP yields the highest reward and the highest risk. In the RTP strategy, the electricity price is influenced by the supply and demand of the energy market. Hence, the energy cost of manufacturing cannot be determined by the manufacturing companies, implying a high level of risk. However, if manufacturing companies seize the opportunity to perform more manufacturing tasks when the energy price is low, the cost-savings will be significant, implying a high level of reward. In this study, we propose an integrated energy management system (IEMS) to reduce the energy cost of manufacturing systems. The IEMS consists of an energy storage equipment and an intelligent switch mechanism. When the electricity price is high, the manufacturing system is powered by the energy storage equipment. When the electricity price is low, the manufacturing system is powered by the public electricity grid, and the energy storage equipment is charged. The decision-making of these operations is performed by the intelligent switch mechanism based on double deep Q-learning. To validate this framework, a case study is conducted, in which an IEMS is developed to reduce the electricity cost of a scale-model factory. Based on an online test of the IEMS in different manufacturing cycles, it is concluded that the proposed IEMS approach achieves a cost reduction of approximately 57.21 %.