A Non-Monotone Line-Search Method for Minimizing Functions with Spurious Local Minima

Abstract: In this paper, we propose a new non-monotone line-search method for smooth unconstrained optimization problems with objective functions that have many non-global local minimizers. The method is based on a relaxed Armijo condition that allows a controllable increase in the objective function between consecutive iterations. This property helps the iterates escape from nearby local minimizers in the early iterations. For objective functions with Lipschitz continuous gradients, we derive worst-case complexity estimates on the number of iterations needed for the method to find approximate stationary points. Numerical results are presented, showing that the new method can significantly outperform other non-monotone methods on functions with spurious local minima.