Online Dictionary Learning with Group Structure Inducing Norms

Thanks to the several successful applications, sparse signal representation has become one of the most actively studied research areas in machine learning. In the sparse coding framework one approximates the observations with the linear combination of a few vectors (basis elements) from a fixed dictionary (Tropp & Wright, 2010). The general sparse coding problem, i.e., the l0-norm solution that searches for the least number of basis elements, is NP-hard. To overcome this difficulty, a popular approach is to apply lp (0 < p ≤ 1) relaxations. The p = 1 special case, the Lasso problem, has become particularly popular since in this case the relaxation leads to a convex problem.