A Survey on Reinforcement Learning based Adaptive Optimal Control Design

Offline solutions to optimal control problems are typically found using the Hamilton-Jacobii-Bellman (HJB) Equation for nonlinear systems and the Algebraic Riccati Equation (ARE) for linear time-invariant (LTI) systems. For the ARE/HJB problem to be solved, system parameters recognition is essential. Optimal control design for a system with partially/completely unknown dynamics is a challenging task. Although the adaptive control design is useful technique for dealing with parameter uncertainty, it does not ensure optimality. Adaptive optimum controllers (AOC) have been developed to fill the gap between adaptive and optimal control. The idea behind AOC is to create controllers that can estimate the system and control parameters in real-time while preserving the optimal performance of the obtained control policy. This paper will carry out an extensive literature review on the previous work done by the different researchers in the field of AOC designs. This paper does not only provide a detailed literature review on AOC, but it also provides a detailed analysis of principal of reinforcement learning based techniques in designing adaptive optimal controllers and future scope of the RL-based AOC design techniques. Finally, a simulation results is shown to validate one of the reinforcement algorithm Q-techniques for LTI system.