Extending Gaussian process emulation using cluster analysis and artificial neural networks to fit big training sets

Gaussian process (GP) emulation is a relatively recent statistical technique that provides a fast-running approximation to a complex computer model, given training data generated by the considered model. Despite its sound theoretical foundation, GP emulation falls short in practical applications where the training dataset is very large, due to numerical instabilities in inverting the correlation matrix. We show how GP emulation can be extended to handle large training sets by first dividing the training set into smaller subsets using cluster analysis, then training an emulator for each subset, and finally combining the emulators using an artificial neural network (ANN). Our work has also conceptual relevance, as it shows how to solve a big data problem by introducing a local level in input space, where each emulator specialises in a certain subregion, and a global level, where the identified local features of the computer model are combined into a global view.