Robust Gaussian Process Regression with Input Uncertainty: A PAC-Bayes Perspective

The Gaussian process (GP) algorithm is considered as a powerful nonparametric-learning approach, which can provide uncertainty measurements on the predictions. The standard GP requires clearly observed data, unexpected perturbations in the input may lead to learned regression model mismatching. Besides, GP also suffers from the lack of good generalization performance guarantees. To deal with data uncertainty and provide a numerical generalization performance guarantee on the unknown data distribution, this article proposes a novel robust noisy input GP (NIGP) algorithm based on the probably approximately correct (PAC) Bayes theory. Furthermore, to reduce the computational complexity, we develop a sparse NIGP algorithm, and then develop a sparse PAC-Bayes NIGP approach. Compared with NIGP algorithms, instead of maximizing the marginal log likelihood, one can optimize the PAC-Bayes bound to pursue a tighter generalization error upper bound. Experiments verify that the NIGP algorithms can attain greater accuracy. Besides, the PAC-NIGP algorithms proposed herein can achieve both robust performance and improved generalization error upper bound in the face of both uncertain input and output data.