EXACT PENALTY FUNCTION FOR \ell2,1 NORM MINIMIZATION OVER THE STIEFEL MANIFOLD

\ell 2,1 norm minimization with orthogonality constraints, which comprise a feasible region called the Stiefel manifold, has wide applications in statistics and data science. The state-ofthe-art approaches adopt a proximal gradient technique on either the Stiefel manifold or its tangent spaces. The consequent subproblem does not have a closed-form solution and hence requires an iterative procedure to solve, which is usually time-consuming. In this paper, we discover that the Lagrangian multipliers of the orthogonality constraints in this class of problems are of closed-form expressions. By using this closed-form expression, we introduce a penalty function for this type of problem. We theoretically demonstrate the equivalence between the penalty function and the original \ell 2,1 norm minimization under mild assumptions. Based on the exact penalty function, we propose an inexact proximal gradient method in which the subproblem is of closed-form solution. The global convergence and the worst case complexity are established. Numerical experiments illustrate the advantages of our method when compared with the existing proximal-based first-order methods.