Low-Rank Matrix Approximations Do Not Need a Singular Value Gap

Low-rank approximations to a real matrix A can be conputed from ZZ(T)A, where Z is a matrix with orthonormal columns, and the accuracy of the approximation can be estimated from some norm of A-ZZ(T)A. We show that computing A-ZZ(T)A in the two-norm, Frobenius norms, and more generally any Schatten p-norm is a well-posed mathematical problem; and, in contrast to dominant subspace computations, it does not require a singular value gap. We also show that this problem is well-conditioned (insensitive) to additive perturbations in A and Z, and to dimensionchanging or multiplicative perturbations in A-regardless of the accuracy of the approximation. For the special case when A does indeed have a singular values gap, connections are established between low-rank approximations and subspace angles.