Parallelization of a Neural Network Algorithm for Handwriting Recognition - Can we Increase the Speed, Keeping the Same Accuracy.

This paper examines the problem of parallelizing neural network training using the back-propagation neural network, as a breakthrough example in the field of deep learning. The challenge of our solution is to twist the algorithm in such a way so it can be executed in parallel, rather than sequentially. In this paper, we test validity of a research hypothesis if the speed can be increased by parallelizing the back-propagation algorithm and keep the same accuracy. For this purpose, we developed a use-case of a handwriting recognition algorithm and conducted several experiments to test the performance, both in execution speed and accuracy. At the end, we examine just how much it benefits to write a parallel program for a neural network, with regards to the time it takes to train the neural network and the accuracy of the predictions. Our handwriting problem is that of classification, and in order to implement any sort of solution, we must have data. The MNIST dataset of handwritten digits provides necessary data to solve the problem.