Improving Quasi-Newton Methods via Image and Projection Operators

Abstract: Designing efficient quasi-Newton methods is an important problem in nonlinear optimization and the solution of systems of nonlinear equations. From the perspective of the matrix approximation process, this paper presents a unified framework for establishing the quadratic termination property that covers the Broyden family, the generalized PSB family, and good Broyden method. Based on this framework, we employ operators to map the correction direction $s_k$ in the quasi-Newton equation to a specific subspace, which ensures quadratic termination for these three classes of methods without relying on exact line searches. We derive the corresponding image and projection operators, analyze their improved properties in matrix approximation, and design practical algorithms accordingly. Preliminary numerical results show that the proposed operator-based methods yield significant improvements in the performance of the Davidon-Fletcher-Powell (DFP), Broyden-Fletcher-Goldfarb-Shanno (BFGS), Powell-Symmetric-Broyden (PSB), limited-memory BFGS (L-BFGS) and Broyden's ``good'' methods (BGM).