Random multi-block ADMM: an ALM based view for the QP case

Abstract: Embedding randomization procedures in the Alternating Direction Method of Multipliers (ADMM) has recently attracted an increasing amount of interest as a remedy to the fact that the direct multi-block generalization of ADMM is not necessarily convergent. Even if, in practice, the introduction of such techniques could \textit{mitigate} the diverging behaviour of the multi-block extension of ADMM, from the theoretical point of view, it can ensure just the \textit{convergence in expectation}, which may not be a good indicator of its robustness and efficiency. In this work, analysing the strongly convex quadratic programming case, we interpret the block Gauss-Seidel sweep performed by the multi-block ADMM in the context of the inexact Augmented Lagrangian Method. Using the proposed analysis, we are able to outline an alternative technique to those present in literature which, supported from stronger theoretical guarantees, is able to ensure the convergence of the multi-block generalization of the ADMM method.