Alternating direction algorithms for $\ell_0$ regularization in compressed sensing

Abstract: In this paper we propose three iterative greedy algorithms for compressed sensing, called \emph{iterative alternating direction} (IAD), \emph{normalized iterative alternating direction} (NIAD) and \emph{alternating direction pursuit} (ADP), which stem from the iteration steps of alternating direction method of multiplier (ADMM) for $\ell_0$-regularized least squares ($\ell_0$-LS) and can be considered as the alternating direction versions of the well-known iterative hard thresholding (IHT), normalized iterative hard thresholding (NIHT) and hard thresholding pursuit (HTP) respectively. Firstly, relative to the general iteration steps of ADMM, the proposed algorithms have no splitting or dual variables in iterations and thus the dependence of the current approximation on past iterations is direct. Secondly, provable theoretical guarantees are provided in terms of restricted isometry property, which is the first theoretical guarantee of ADMM for $\ell_0$-LS to the best of our knowledge. Finally, they outperform the corresponding IHT, NIHT and HTP greatly when reconstructing both constant amplitude signals with random signs (CARS signals) and Gaussian signals.