Alternating Optimization for Tensor Factorization with Orthogonality Constraints: Algorithm and Parallel Implementation

We consider the problem of tensor factorization in the cases where one of the factors is constrained to have orthonormal columns. We adopt the alternating optimization framework and derive an efficient algorithm that is also suitable for parallel implementation. We describe in detail a distributed memory implementation of the algorithm on a three-dimensional processor grid. The speedup attained by a message-passing implementation of the algorithm is significant, indicating that it is a competitive candidate for the solution of very large tensor factorization problems with orthogonality constraints.