Density Estimation in High and Ultra High Dimensions, Regularization, and the L1 Asymptotics ?

This article gives a theoretical treatment of the asymptotics of the L1 error of a model-based estimate of a density f(xjµ) on a flnite dimensional Euclidean space R k . The dimension p of the parameter vector µ is considered arbitrary but flxed in Section 2. Two theorems in Section 2 lay out the weak limits of a suitably scaled L1 error, with a general estimating sequence ^ µ and a general family of smooth densities f(xjµ) dominated by some æ-flnite measure, the discrete case included. We show that the L1 error converges at the coarsest rate corresponding to the difierent coordinates of the parameter vector µ. Four applications are detailed in Section 3, a special one being a new confldence interval for a Poisson mean. Section 4 considers the high and the ultra high dimensional case, where p grows with n. The exact critical growth rate for p when maximum likelihood starts to falter is derived. Maximum likelihood is shown to exhibit a trichotomy of behavior; the desired behavior below the threshold, problematic behavior at the threshold, and disastrous behavior above the threshold. It is then shown that regularization, if coupled with the right amount of sparsity, can return consistent density estimation, even at the best possible n ¡1=2 rate. We give a complete description of the limiting behavior of the regularized density estimate under difierent sparsity conditions. Section 4 is specialized to the Gaussian case due to its special importance and well known links to function estimation.