A Unified Decentralized Nonconvex Algorithm under Kurdyka-Łojasiewicz Property

Abstract: In this paper, we study the decentralized optimization problem of minimizing a finite sum of continuously differentiable and possibly nonconvex functions over a fixed-connected undirected network. We propose a unified decentralized nonconvex algorithmic framework that subsumes existing state-of-the-art gradient tracking algorithms and particularly several quasi-Newton algorithms. We present a general analytical framework for the convergence of our unified algorithm under both nonconvex and the Kurdyka-Łojasiewicz condition settings. We also propose some quasi-Newton variants that fit into our framework, where economical implementation strategies are derived for ensuring bounded eigenvalues of Hessian inverse approximations. Our numerical results show that these newly developed algorithms are very efficient compared with other state-of-the-art algorithms for solving decentralized nonconvex smooth optimization.