Modeling and Deep Reinforcement Learning Based Control Parameter Tuning for Voltage Source Converter in a Renewable Energy Generation System

The fast response and low inertia characteristics of converter-based generation (CBG) lead to a new stability issue that limits renewable energy development. This is due to the fact that, using traditional control theory as basic, there is no unified and effective way to linearize the electronic device and set the parameters in the existing analysis method. In order to optimize the parameters tuning process of the converter control, a benchmark model is adopted in this paper, and the linearization model is updated by selecting suitable variables and detailed with considering the influence of resistance of the converter. Based on this model, the stability margin related to the parameters of the system is analyzed. Furthermore, to consider more dynamics and make the converter controller focus on the topologies and scenarios, the dynamic response of a disturbance in the grid is selected as the iteration state to design a controller using Deep Reinforcement Learning (DRL). That is, the cascaded voltage and current control along with droop control are tuned by Deep Deterministic Policy Gradient (DDPG) algorithm in the linearized model. The validation tests are carried out on the nonlinear model in Simulink test and real-time platform. They indicate that the proposed linearization and tuning method provides more accuracy and stability for the power system in various grid conditions.