Convergence of Limited Communications Gradient Methods

Abstract: Distributed optimization increasingly plays a central role in economical and sustainable operation of cyber-physical systems. Nevertheless, the complete potential of the technology has not yet been fully exploited in practice due to communication limitations posed by the real-world infrastructures. This work investigates fundamental properties of distributed optimization based on gradient methods, where gradient information is communicated using limited number of bits. In particular, a general class of quantized gradient methods are studied where the gradient direction is approximated by a finite quantization set. Sufficient and necessary conditions are provided on such a quantization set to guarantee that the methods minimize any convex objective function with Lipschitz continuous gradient and a nonempty and bounded set of optimizers. A lower bound on the cardinality of the quantization set is provided, along with specific examples of minimal quantizations. Convergence rate results are established that connect the fineness of the quantization and the number of iterations needed to reach a predefined solution accuracy. Generalizations of the results to a relevant class of constrained problems using projections are considered. Finally, the results are illustrated by simulations of practical systems.