Reinforcement learning solution for HJB equation arising in constrained optimal control problem
Neural Networks(2015)
摘要
The constrained optimal control problem depends on the solution of the complicated Hamilton-Jacobi-Bellman equation (HJBE). In this paper, a data-based off-policy reinforcement learning (RL) method is proposed, which learns the solution of the HJBE and the optimal control policy from real system data. One important feature of the off-policy RL is that its policy evaluation can be realized with data generated by other behavior policies, not necessarily the target policy, which solves the insufficient exploration problem. The convergence of the off-policy RL is proved by demonstrating its equivalence to the successive approximation approach. Its implementation procedure is based on the actor-critic neural networks structure, where the function approximation is conducted with linearly independent basis functions. Subsequently, the convergence of the implementation procedure with function approximation is also proved. Finally, its effectiveness is verified through computer simulations.
更多查看译文
关键词
Constrained optimal control,Data-based,Off-policy reinforcement learning,Hamilton–Jacobi–Bellman equation,The method of weighted residuals
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络