Optimal Linear Estimation Under Unknown Nonlinear Transform

Linear regression studies the problem of estimating a model parameter beta* is an element of R-p, from n observations {(y(i), X-i)}(i=1)(n) from linear model y(i) = < X-i, beta*> + is an element of(i). We consider a significant generalization in which the relationship between < X-i, beta*> and y(i) is noisy, quantized to a single bit, potentially nonlinear, noninvertible, as well as unknown. This model is known as the single-index model in statistics, and, among other things, it represents a significant generalization of one-bit compressed sensing. We propose a novel spectral-based estimation procedure and show that we can recover beta* in settings (i.e., classes of link function f) where previous algorithms fail. In general, our algorithm requires only very mild restrictions on the (unknown) functional relationship between y(i) and < X-i, beta*>. We also consider the high dimensional setting where beta* is sparse, and introduce a two-stage nonconvex framework that addresses estimation challenges in high dimensional regimes where p >> n. For a broad class of link functions between < X-i, beta*> and y(i), we establish minimax lower bounds that demonstrate the optimality of our estimators in both the classical and high dimensional regimes.