Proportional Volume Sampling and Approximation Algorithms for A-Optimal Design.

We study the A-optimal design problem where we are given vectors v1, ..., vn ∈ Rd, an integer k ≥ d, and the goal is to select a set S of k vectors that minimizes the trace of (Σi∈SviviT)−1. Traditionally, the problem is an instance of optimal design of experiments in statistics [35] where each vector corresponds to a linear measurement of an unknown vector and the goal is to pick k of them that minimize the average variance of the error in the maximum likelihood estimate of the vector being measured. The problem also finds applications in sensor placement in wireless networks [22], sparse least squares regression [8], feature selection for k-means clustering [9], and matrix approximation [13, 14, 5]. In this paper, we introduce proportional volume sampling to obtain improved approximation algorithms for A-optimal design. Given a matrix, proportional volume sampling involves picking a set of columns S of size k with probability proportional to μ(S) times det(Σi∈SviviT) for some measure μ. Our main result is to show the approximability of the A-optimal design problem can be reduced to approximate independence properties of the measure μ. We appeal to hard-core distributions as candidate distributions μ that allow us to obtain improved approximation algorithms for the A-optimal design. Our results include a d-approximation when k = d, an (1 + ϵ)-approximation when [MATH HERE] and [MATH HERE]-approximation when repetitions of vectors are allowed in the solution. We also consider generalization of the problem for k ≤ d and obtain a k-approximation. We also show that the proportional volume sampling algorithm gives approximation algorithms for other optimal design objectives (such as D-optimal design [36] and generalized ratio objective [27]) matching or improving previous best known results. Interestingly, we show that a similar guarantee cannot be obtained for the E-optimal design problem. We also show that the A-optimal design problem is NP-hard to approximate within a fixed constant when k = d.