A continuous-time model to interpolate between speed and function value restart in accelerated first order methods

Abstract: We introduce a new restarting scheme for a continuous inertial dynamics with Hessian driven-damping, and establish a linear convergence rate for the function values along the restarted trajectories. The proposed routine is a generalization of existing speed restart schemes, and interpolates between speed and function value restarts, considerably delaying the restarting time, while preserving the function value decrease. Numerical experiments show an improvement in the convergence rates for both continuous-time dynamical systems, and associated first-order algorithms derived via time discretization.