Probabilistic fusion of crowds and experts for the search of gravitational waves

The acquisition of training labels in machine learning classification tasks is expensive. In the last years, crowdsourcing has emerged as a popular approach to label a training set. Crowdsourcing shares the labeling effort among a large number of (possibly non-expert) annotators. Moreover, in many realistic applications, a limited number of expert labels can also be collected to complement the crowdsourcing ones. Such combination of (millions of) crowdsourced and (a few) expert labels is precisely the setting in the GravitySpy project. The goal of GravitySpy is to enhance the detection of gravitational waves, which provide a new way of exploring the early universe in astrophysics (their first detection got the 2017 Physics Nobel prize). In this work, we propose a new probabilistic crowdsourcing model based on sparse Gaussian Processes (GPs) which allows for the integration of expert labels. To the best of our knowledge, this is the first probabilistic GP-based method that tackles this setting. We demonstrate that the resulting objective function to be optimized is a natural fusion of the crowdsourcing and the standard sparse GP classification objectives. Desirable theoretical properties of the crowdsourcing method, translate in a mathematical sound manner into the new method. The new algorithm is implemented in TensorFlow. A controlled experiment illustrates the properties and behavior of the proposed method. We also show that it performs as theoretically expected in a well-known real-world crowdsourcing dataset. Finally, its application to GravitySpy obtains 92.58% overall accuracy and 92.27% test-likelihood, outperforming all previous methods in the literature.(c) 2022 Elsevier B.V. All rights reserved.