Characterizing Visual Representations within Convolutional Neural Networks : Toward a Quantitative Approach

Even when parameters of a deep neural network are fully known, it is still not always clear how and why a given network “works,” and how neurons in the network contribute to overall network performance. In this paper, we propose a suite of tools for visualizing and characterizing deep networks that aims to reveal their key representation properties. We present preliminary results with a large collection of randomly generated networks to explore how representation properties relate to network depth and performance, and we find that these properties can explain a substantial fraction of variation in overall network performance.