Higher Degree Inexact Model for Optimization problems

Abstract: In this paper, it was proposed a new concept of the inexact higher degree $(\delta, L, q)$-model of a function that is a generalization of the inexact $(\delta, L)$-model, $(\delta, L)$-oracle and $(\delta, L)$-oracle of degree $q \in [0,2)$. Some examples were provided to illustrate the proposed new model. Adaptive inexact gradient and fast gradient methods for convex and strongly convex functions were constructed and analyzed using the new proposed inexact model. A universal fast gradient method that allows solving optimization problems with a weaker level of smoothness, among them non-smooth problems was proposed. For convex optimization problems it was proved that the proposed gradient and fast gradient methods could be converged with rates $O\left(\frac{1}{k} + \frac{\delta}{k^{q/2}}\right)$ and $O\left(\frac{1}{k^2} + \frac{\delta}{k^{(3q-2)/2}}\right)$, respectively. For the gradient method, the coefficient of $\delta$ diminishes with $k$, and for the fast gradient method, there is no error accumulation for $q \geq 2/3$. It proposed a definition of an inexact higher degree oracle for strongly convex functions and a projected gradient method using this inexact oracle. For variational inequalities and saddle point problems, a higher degree inexact model and an adaptive method called Generalized Mirror Prox to solve such class of problems using the proposed inexact model were proposed. Some numerical experiments were conducted to demonstrate the effectiveness of the proposed inexact model, we test the universal fast gradient method to solve some non-smooth problems with a geometrical nature.