On the Acceleration of Proximal Bundle Methods

Abstract: The proximal bundle method (PBM) is a fundamental and computationally effective algorithm for solving nonsmooth optimization problems. In this paper, we present the first variant of the PBM for smooth objectives, achieving an accelerated convergence rate of $O(\epsilon^{-1/2}\log(1/\epsilon))$, where $\epsilon$ is the desired accuracy. Our approach addresses an open question regarding the convergence guarantee of proximal bundle type methods, which was previously posed in two papers. We interpret the PBM as a proximal point algorithm and base our proposed algorithm on an accelerated inexact proximal point scheme. Our variant introduces a novel null step test and oracle while maintaining the core structure of the original algorithm. The newly proposed oracle substitutes the traditional cutting planes with a smooth lower approximation of the true function. We show that this smooth interpolating lower model can be computed as a convex quadratic program. We also examine a second setting where Nesterov acceleration can be effectively applied, specifically when the objective is the sum of a smooth function and a piecewise linear one.