Randomized Iterative Methods with Alternating Projections

Abstract: We use a unified framework to summarize sixteen randomized iterative methods including Kaczmarz method, coordinate descent method, etc. Some new iterative schemes are given as well. Some relationships with \textsc{mg} and \textsc{ddm} are also discussed. We analyze the convergence properties of the iterative schemes by using the matrix integrals associated with alternating projectors, and demonstrate the convergence behaviors of the randomized iterative methods by numerical examples.