Even Simpler Deterministic Matrix Sketching

This paper provides a one-line proof of Frequent Directions (FD) for sketching streams of matrices. It simplifies the main results in [1] and [2]. The simpler proof arises from sketching the covariance of the stream of matrices rather than the stream itself. Introduction Let Xt ∈ R d×nt be a stream of matrices. Let C = ∑ T t=1 XtX T t ∈ R d×d be their covariance matrix. Frequent Directions [1] maintains a rank deficient approximate covariance matrix C̃t ∈ R d×d using Algorithm 1. Set C̃0 ∈ R d×d to be the all zeros matrix. Then, at time t = 1, . . . , T compute C̃t = Update(C̃t−1, Xt, l). Algorithm 1 Frequent Directions (FD) Update 1: function Update(C̃t−1, Xt, l) 2: UtΛtU T t = C̃t−1 +XtX T t 3: return C̃t = U ·max(Λ − I · λ t l , 0) · U 4: end function Above, UtΛtU T t is the eigen-decomposition of C̃t−1+XtX T t and λ l is the its l’th largest eigenvalue. Note that the rank of C̃t is at most l − 1 for all t by construction. It can therefore be stored in O(dl) space. Assuming nt < l, the update operation itself also consumes at most O(dl) space. Lemma 1 (simplified from [2] and [1]). Let C̃ denote the approximated covariance produced by FD and λi be the eigenvalues of the exact covariance C in descending order. For any l and simultaneously for all k < l we have ‖C − C̃‖ ≤ 1 l− k d