Optimal trajectory tracking control for a class of nonlinear nonaffine systems via generalized N‐step value gradient learning

In this paper, the tracking control problem of unknown nonlinear systems is solved by using the generalized N-step value gradient learning algorithm with parameter lambda [GNSVGL(lambda)]. The GNSVGL(lambda) algorithm can provide optimal tracking decisions faster than traditional ones. Initialized by different positive semi-definite functions, the monotonicity and convergence properties of the proposed algorithm are proven. Under some conditions, the stability analysis of the value-iteration-based algorithm is provided. The one-return and lambda-return critic neural networks are constructed to approximate the gradient of the one-return and lambda-return cost functions. The action neural network is employed to approximate the control law of the error system. It is emphasized that one-return and lambda-return critic networks are combined to train the action neural network. Finally, via conducting simulation studies and comparisons, the excellent tracking performance of the proposed algorithm is confirmed.