A Kernel-Based Sarsa( \lambda ) Algorithm with Clustering-Based Sample Sparsification.

In the past several decades, as a significant class of solutions to the large scale or continuous space control problems, kernel-based reinforcement learning (KBRL) methods have been a research hotspot. While the existing sample sparsification methods of KBRL exist the problems of low time efficiency and poor effect. For this problem, we propose a new sample sparsification method, clustering-based novelty criterion (CNC), which combines a clustering algorithm with a distance-based novelty criterion. Besides, we propose a clustering-based selective kernel Sarsa(lambda) (CSKS(lambda)) on the basis of CNC, which applies Sarsa(lambda) to learning parameters of the selective kernel-based value function based on local validity. Finally, we illustrate that our CSKS(lambda) surpasses other state-of-the-art algorithms by Acrobot experiment.