Model-free intelligent critic design with error analysis for neural tracking control

The core of the optimal tracking control problem for nonlinear systems is how to ensure that the controlled system tracks the desired trajectory. The utility functions in previous studies have different properties which affect the final tracking effect of the intelligent critic algorithm. In this paper, we introduce a novel utility function and propose a Q -function based policy iteration algorithm to eliminate the final tracking error. In addition, neural networks are used as function approximator to approximate the performance index and control policy. Considering the impact of the approximation error on the tracking performance, an approximation error bound for each iteration of the novel Q -function is established. Under the given conditions, the approximation Q -function converges to the finite neighborhood of the optimal value. Moreover, it is proved that weight estimation errors of neural networks are uniformly ultimately bounded. Finally, the effectiveness of the algorithm is verified by the simulation example.