Manifold Quadratic Penalty Alternating Minimization for Sparse Principal Component Analysis

Abstract: Optimization on the Stiefel manifold or with orthogonality constraints is an important problem in many signal processing and data analysis applications such as Sparse Principal Component Analysis (SPCA). Algorithms such as the Riemannian proximal gradient algorithms addressing this problem usually involve an intricate subproblem requiring a semi-smooth Newton method hence, simple and effective operator splitting methods extended to the manifold setting such as the Alternating Direction Method of Multipliers (ADMM) have been proposed. However, another simple operator-splitting method, the Quadratic Penalty Alternating Minimization (QPAM) method which has been successful in image processing to our knowledge, has not yet been extended to the manifold setting. In this paper, we propose a manifold QPAM (MQPAM) which is very simple to implement. The iterative scheme of the MQPAM consists of a Riemannian Gradient Descent (RGD) subproblem and a subproblem in the form of a proximal operator which has a closed-form solution. Experiments on the SPCA problem show that our proposed MQPAM is at par with or better than several other algorithms in terms of sparsity and CPU time.