Algorithmic generalization ability of PALM for double sparse regularized regression

We propose a novel double sparse regularized modelling paradigm for Generalized Linear Model in high-dimensional setting, where we allow the underlying population distribution to be possibly heavy-tailed or heteroscedastic. Differing from the existing approaches that seek to find a robust estimate against potential heterogeneity in the training samples via robustification mechanisms, the proposed regularized modelling paradigm can identify and quantify the heterogeneous-specific effect simultaneously, and it contributes to a more consistent and optimal estimate, and is flexible enough to accommodate latent heterogeneity both in individual and subgroup levels. The proposed method has three popular applications, i.e., heterogeneous analysis, outlier detection and image restoration. We devise an efficient learning algorithm, referring to Proximal Alternating Linearized Minimization (PALM), to implement the proposed approach. The PALM algorithm proceeds by leveraging Linearization and Alternation Minimization techniques, and works well in general regularized (possibly nonconvex) bi-block optimizations. We discuss the Algorithmic Generalization Ability of the PALM for the regularized generalized linear regression, which demonstrates the asymptotic consistency between iterative sequences and true parameter vectors with a high probability guarantee. The computational efficiency, performance in estimation, and predictive validation are empirically verified with several simulations and real data applications, and results indicate that the proposed approach is competitive with the existing state-of-the-art methods.