Some recent results on accelerated parallel projection methods

We present some inter-related results on accelerated parallel projection methods. The first result is called the Cimmino-Kaczmarz Equivalence, meaning that the Cimmino algorithm, which consists of projections and vector averaging, is equivalent to the Kaczmarz algorithm (which uses only projections) in some superspace of the problem space. The practical consequence of this is that in the CARP-CG algorithm [G & G, PARCO 2010], the internal Kaczmarz processing of the subdomains can be replaced by Cimmino, which is more amenable to fast computing on a GPU. The Kaczmarz-Cimmino equivalence implies a formal convergence proof for such a modification. This result is quite general, allowing different relaxation parameters to be used in the Cimmino algorithm. A second result concerns the CGMN algorithm [Björck & Elfving, BIT 1979] and its blockparallel version CARP-CG. Both are examples of the Kaczmarz algorithm accelerated by CG, as follows: by running Kaczmarz in a forward and backward sweep, the resulting iteration matrix is symmetric and positive semi-definite, so the process can be accelerated by CG. Kaczmarz is, in fact SOR ran on AAT , where A is the normalized system matrix. Although the use of AAT is generally not recommended (because its condition number is the square of the condition number of A), it has been shown in previous work that after A is normalized, the diagonal elements of AAT are all 1, and the off-diagonal elements are < 1. Our new result uses two model problems a convection-dominated elliptic PDE and a high-frequency Helmholtz equation to demonstrate that when a certain problematic parameter increases, the maximal off-diagonal element of a row of A increases unboundedly w.r.t. the diagonal, while in AAT , the maximal off-diagonal element remains well-bounded below the diagonal. The ”problematic parameter” is the size of the convection term in the elliptic PDE, and the frequency in the Helmholtz equation. A third result concerns a major issue in domain decomposition (DD): the problem of eliminating inaccuracies caused by integrating the subdomain solutions across subdomain boundaries. An even harder problem arises in the case of cross points, at which three or more subdomains meet. This topic has received a lot of attention in recent years, with several problemspecific solutions. It is shown, and formally proved, that these problems do not exist with the CARP-CG algorithm. This is due to the fact that in CARP-CG, both the local processing and the merging of the local solutions are actually solved in a certain superspace in a unified manner. Furthermore, there is no need for any problem-specific adaptation. The concept of componentaveraged DD (CADD) generalizes CARP-CG by allowing the use of other methods besides Kaczmarz or Cimmino for the internal processing in the subdomains. Sufficient conditions for the convergence of a CADD method are discussed.