“I can tell you what it’s not”: active learning from counterexamples

When dealing with feedback from a human expert in a classification process, we usually think of obtaining the correct class label for an example. However, in many real-world settings, it may be much easier for the human expert to tell us to which classes the example does not belong. We propose a framework for this very practical setting to incorporate this kind of feedback. We demonstrate empirically that stable classification models can be built even in the case of partial not-label information and introduce a method to select useful training examples.