GeNIOS: an (almost) second-order operator-splitting solver for large-scale convex optimization

Abstract: We introduce the GEneralized Newton Inexact Operator Splitting solver (GeNIOS) for large-scale convex optimization. GeNIOS speeds up ADMM by approximately solving approximate subproblems: it uses a second-order approximation to the most challenging ADMM subproblem and solves it inexactly with a fast randomized solver. Despite these approximations, GeNIOS retains the convergence rate of classic ADMM and can detect primal and dual infeasibility from the algorithm iterates. At each iteration, the algorithm solves a positive-definite linear system that arises from a second-order approximation of the first subproblem and computes an approximate proximal operator. GeNIOS solves the linear system using an indirect solver with a randomized preconditioner, making it particularly useful for large-scale problems with dense data. Our high-performance open-source implementation in Julia allows users to specify convex optimization problems directly (with or without conic reformulation) and allows extensive customization. We illustrate GeNIOS's performance on a variety of problem types. Notably, GeNIOS is more than ten times faster than existing solvers on large-scale, dense problems.