Numerical Linear Algebra In The Streaming Model

We give near-optimal space bounds in the streaming model for linear algebra problems that include estimation of matrix products, linear regression, low-rank approximation, and approximation of matrix rank. In the streaming model, sketches of input matrices are maintained under updates of matrix entries; we prove results for turnstile updates, give in an arbitrary order. We give the first lower bounds known for the space needed by the sketches, for a given estimation error epsilon. We sharpen prior upper bounds, with respect to combinations of space, failure probability, and number of passes. The sketch we use for matrix A is simply S-T A, where S is a sign matrix.Our results include the following upper and lower bounds on the bits of space needed for 1-pass algorithms. Here A is an n x d matrix, B is an n x d' matrix, and c := d + d'. These results are give for fixed failure probability; for failure probability delta > 0, the upper bounds require a factor of log (1/delta) more space. We assume the inputs have integer entries specified by O(log(nc)) bits, or O(log(nd)) bits.1. (Matrix Product) Output matrix C withparallel to A(T) B - C parallel to <= epsilon parallel to A parallel to parallel to B parallel toWe show that Theta(c epsilon(-2) log(nc)) space is needed.2. (Linear Regression For d' = 1, so that B is a vector b, find x so thatparallel to Ax - b parallel to <= (1 + epsilon) min(x' is an element of IRd)parallel to Ax' - b parallel to.We show that Theta(d(2)epsilon(-1) log (nd)) space is needed.3. (Rank-k Approximation) Find matrix (A) over tilde (k) of rank no more than k, so thatparallel to A - (A) over tilde (k)parallel to <= (1 + epsilon)parallel to A - A(k)parallel to,where A(k) is the best rank-k approximation to A. Our lower bound is Omega(k epsilon(-1)(n + d) log(nd)) space, and we give a one-pass algorithm matching this when A is given row-wise or column-wise. For general updates, we give a one-pass algorithm needingO(k epsilon(-2)(n + d/epsilon(2)) log(nd))space. We also give upper and lower bounds for algorithms using multiple passes, and a sketching analog of the CUR decomposition.