Detecting adversarial attacks through neural activations

This paper presents a methodology to detect adversarial manipulations to image data by examining the activation patterns in a neural network. By comparing a sample’s layer-wise neural activations to clusters of its predicted class, we are able to detect irregularities between a model’s layer activations and its final predicted class. We evaluate our detection method using the FGSM attack method as well as the Carlini-Wagner L2 attack and misclassified images from the ImageNet dataset.