DNN Feature Map Gram Matrices for Mitigating White-Box Attacks on Image Classification

Deep convolutional neural networks accurately classify a diverse range of natural images, but may be easily deceived when designed, imperceptible perturbations are embedded in the images. In this paper, we design a multi-pronged training, input transformation, and image ensemble system that is attack agnostic and not easily estimated. Our system incorporates two novel features. The first is a transformation layer that iteratively filters the input image’s feature maps based on their Gram matrix representational differences to those of randomly sampled training instances from each class, which creates an ensemble of filtered input image copies. The second is a classification system that treats this filtered ensemble as a voting committee that focuses on only those predictions that changed from the original prediction on the unfiltered input image. Our evaluations on the CIFAR10 dataset show our system improves the robustness of an undefended network against a variety of bounded and unbounded white-box attacks under different distance metrics, while sacrificing little accuracy on clean images. Against adaptive full-knowledge attackers creating end-to-end attacks, our system successfully augments the existing robustness of adversarially trained networks, for which our methods are most effectively applied.