Discussion of “concentration for (regularized) empirical risk minimization” by Sara van de Geer and Martin Wainwright

Sara van de Geer and Martin Wainwright combine astute convexity arguments and concentration inequalities for suprema of empirical processes to establish generic concentration inequalities for excess penalized risk. This note discusses possible refinements and extensions. In the Gaussian sequence model, concentration of reconstruction error is likely to be improvable and might depend on the effective sparsity of the typical penalized estimator. In the general setting, concentration of excess penalized risk should be complemented by concentration of empirical excess penalized risk. Recent results on penalized least-square estimation pave the way to such a extensions.