Nonlinear control strategies for 3-DOF control moment gyroscope using deep reinforcement learning

Reinforcement learning is a compelling area of research within machine learning because it enables the improvement of control strategies for future manipulation of dynamic systems, leveraging previous data even without a precise model of the system. It usually makes complex, model-free predictions from data alone, which is actually consistent with the purpose of control in that they both aim to design systems using richly structured perceptions to execute planning and control strategies that adequately adapt to changing environments. The robust trajectory tracking control of intricate mechanical systems presents a challenging problem that necessitates effective control methods. In this paper, we propose a novel nonlinear control strategy based on deep reinforcement learning to solve the trajectory tracking problem of a 3-degree-of-freedom (3-DOF) control moment gyroscope (CMG). First, dynamic modeling of the 3-DOF CMG is used as a policy solver for the reinforcement learning training environment, and transfer learning is employed to bridge the reality gap. Then, the hyperparameters and reward functions of the neural network are optimized using the asynchronous successive halving algorithm. Ultimately, the twin delay depth determination policy gradient algorithm is trained in simulation to yield an agent capable of tracking user-defined trajectory routes as a nonlinear controller for the system. Both simulation and experimental results show that the proposed method works well for both high-frequency and low-frequency varying trajectory tracking control, and that the proposed method has better response speed and robustness than classic linear parameter-varying control methods and the state-of-the-art nonlinear parameter-varying method and the neural network-based feedback linearization adaptive control method.