Accelerated canonical polyadic decomposition using mode reduction.

CANonical polyadic DECOMPosition (CANDECOMP, CPD), also known as PARAllel FACtor analysis (PARAFAC) is widely applied to Nth-order (N ≥ 3) tensor analysis. Existing CPD methods mainly use alternating least squares iterations and hence need to unfold tensors to each of their N modes frequently, which is one major performance bottleneck for large-scale data, especially when the order N is large. To overcome this problem, in this paper, we propose a new CPD method in which the CPD of a high-order tensor (i.e., N > 3) is realized by applying CPD to a mode reduced one (typically, third-order tensor) followed by a Khatri-Rao product projection procedure. This way is not only quite efficient as frequently unfolding to N modes is avoided, but also promising to conquer the bottleneck problem caused by high collinearity of components. We show that, under mild conditions, any Nth-order CPD can be converted to an equivalent third-order one but without destroying essential uniqueness, and theoretically they simply give consistent results. Besides, once the CPD of any unfolded lower order tensor is essentially unique, it is also true for the CPD of the original higher order tensor. Error bounds of truncated CPD are also analyzed in the presence of noise. Simulations show that, compared with state-of-the-art CPD methods, the proposed method is more efficient and is able to escape from local solutions more easily.