Parallel Training of Recurrent Neural Networks

We accelerated recurrent neural network (RNN) training on multi-core CPUs using Halide. Our implementation used data parallelism and parallel matrix multiplication operations (Multi-threads + SIMD Vectorization), and achieved approx. 39x speedup against sequential training of the RNN in Halide on a 61 core Xeon Phi CPU. We also compared our optimized Halide implementation with a optimized NumPy implementation of the same algorithm with batching on the Xeon Phi, and report our results.