Dynamic-stride-net: deep convolutional neural network with dynamic stride

It is crucial to reduce the cost of deep convolutional neural networks while preserving their accuracy. Existing methods adaptively prune DNNs in a layer-wise or channel-wise manner based on the input image. In this paper, we develop a novel dynamic network, namely Dynamic-Stride-Net, to improve residual network with layer-wise adaptive strides in the convolution operations. Dynamic-Stride-Net leverages a gating network to adaptively select the strides of convolutional blocks based on the outputs of the previous layer. To optimize the selection of strides, the gating network is trained by reinforcement learning. The floating point operations per second (FLOPS) is significantly reduced by adapting the strides to convolutional layers without loss of accuracy. Dynamic-Stride-Net reduces the computational cost by 35%-50% with equivalent accuracy of the original model on CIFAR-10 and CIFAR-100 datasets. It outperforms the state-of-the-art dynamic networks and static compression methods.