Adversarially Robust Estimate And Risk Analysis In Linear Regression

Adversarially robust learning aims to design algorithms that are robust to small adversarial perturbations on input variables. Beyond the existing studies on the predictive performance to adversarial samples, our goal is to understand the statistical properties of adversarially robust estimates and analyze adversarial risk in the setup of linear regression models. By discovering the statistical minimax rate of convergence of adversarially robust estimators, we emphasize incorporating model information, e.g., sparsity, in adversarially robust learning. Further, we reveal an explicit connection between adversarial and standard estimates and propose a straightforward two-stage adversarial learning framework that facilitates utilizing model structure information to improve adversarial robustness. In theory, the consistency of the adversarially robust estimator is proven and its Bahadur representation is also developed for the statistical inference purpose. The proposed estimator converges in a sharp rate under either a low-dimensional or a sparse scenario. Moreover, our theory confirms two phenomena in adversarially robust learning: adversarial robustness hurts generalization, and unlabeled data improves generalization. In the end, we conduct numerical simulations to verify our theory.