RangeGrad: Explaining Neural Networks by Measuring Uncertainty Through Bound Propagation.

When generating local neural network explanations, many methods remove or obfuscate information at the input and observe the effect on the neural network output. If the lack of certain information causes meaningful changes to the output, we assume it was important and forms part of the explanation for the prediction result. It is not trivial, however, to decide on a clear definition for the absence of information. Previous methods have blurred, darkened or added normally distributed noise to certain portions of the input. In this paper, we propose using interval bounds as a proxy for uncertainty about, or absence of, information. Using this insight, we developed RangeGrad, a novel method for generating saliency maps for neural networks. This method exploits the relationship between uncertainty on the input with prediction uncertainty. We show that the uncertainty framework produces valid explanations in line with existing methods.