Batch Acquisition for Deep Bayesian Active Learning with Imperfect Oracles

Active Learning is used to maintain accuracy while reducing the training set size in many machine learning applications. However active learning approaches are not yet common in practice because they make a strong assumption on the quality of labeled data from an oracle. For machine learning applications whose goal is to estimate a model with very high confidence, we propose a framework for querying data in active learning that works with noisy oracles. In this framework we extend BatchBALD (Kirsch et al., 2019) to create a batch query with a control example and multiple informative examples for the task of deep Bayesian active learning. This allows us to infer the proficiency of the labeler and associate a confidence estimate while using their labels. d