Alternating Minimization, Proximal Minimization and Optimization Transfer Are Equivalent

Abstract: We show that proximal minimization algorithms (PMA), majorization minimization (MM), and alternating minimization (AM) are equivalent. Each type of algorithm leads to a decreasing sequence of objective function. New conditions on PMA are given (the limit of the decreasing sequence of objective function is indeed the infimum of the objective function), which lead to new conditions on AM for the sequence Phi to converge to its infimum. These conditions can then be translated into the language of MM. Examples are given of each type of algorithm and some open questions are posed.