Deflation Methods for Sparse PCA

In analogy to the PCA setting, the sparse PCA problem is often solved by iter- atively alternating between two subtasks: cardinality-co nstrained rank-one vari- ance maximization and matrix deflation. While the former has r eceived a great deal of attention in the literature, the latter is seldom ana lyzed and is typically borrowed without justification from the PCA context. In this work, we demon- strate that the standard PCA deflation procedure is seldom ap propriate for the sparse PCA setting. To rectify the situation, we first develo p several deflation al- ternatives better suited to the cardinality-constrained c ontext. We then reformulate the sparse PCA optimization problem to explicitly reflect th e maximum additional variance objective on each round. The result is a generalized deflation procedure that typically outperforms more standard techniques on real-world datasets.