Recurrent Neural Networks With Intra-Frame Iterations For Video Deblurring

Recurrent neural networks (RNNs) are widely used for sequential data processing. Recent state-of-the-art video deblurringmethods bank on convolutionalrecurrentneural network architectures to exploit the temporal relationship between neighboringframes. In this work, we aim to improve the accuracy of recurrentmodels by adaptingthe hidden states transferredfrom pastframes to the frame being processed so that the relationsbetween video frames could be better used. We iteratively update the hidden state via reusing RNN cell parametersbefore predictingan output deblurredframe. Since we use existing parametersto update the hidden state, our method improves accuracy withoutadditionalmodules. As the architectureremains the same regardless of iteration number fewer iterationmodels can be considered as a partial computationalpath of the models with more iterations.To take advantage of this property, we employ a stochasticmethod to optimize our iterative models better. At training time, we randomly choose the iteration number on the fly and apply a regularizationloss that favors less computation unless there are considerable reconstructiongains. We show that our method exhibits stateof-the-art video deblurringperformance while operatingin real-time speed.