Randomized Kaczmarz iteration methods: Algorithmic extensions and convergence theory

We review and compare several representative and effective randomized projection iteration methods, including the randomized Kaczmarz method, the randomized coordinate descent method, and their modifications and extensions, for solving the large, sparse, consistent or inconsistent systems of linear equations. We also anatomize, extract, and purify the asymptotic convergence theories of these iteration methods, and discuss, analyze, and summarize their advantages and disadvantages from the viewpoints of both theory and computations.