The Geometry of Constrained Optimization: Constrained Gradient Flows via Reparameterization: A-Stable Implicit Schemes, KKT from Stationarity, and Geometry-Respecting Algorithms

Abstract: Gradient-flow (GF) viewpoints unify and illuminate optimization algorithms. Yet most GF analyses focus on unconstrained settings. We develop a geometry-respecting framework for constrained problems by (i) reparameterizing feasible sets with maps whose Jacobians vanish on the boundary (orthant/box) or are rank (n{-}1) (simplex), (ii) deriving flows in the parameter space which induce feasible primal dynamics, (iii) discretizing with A-stable implicit schemes solvable by robust inner loops (Modified Gauss-Newton or a KL-prox (negative-entropy) inner solver), and (iv) proving that stationarity of the dynamics implies KKT-with complementary slackness arising from a simple kinematic mechanism (''null speed'' or ''constant dual speed with vanishing Jacobian''). We also give a Stiefel-manifold treatment where Riemannian stationarity coincides with KKT. These results yield efficient, geometry-respecting algorithms for each constraint class. We include a brief A-stability discussion and present numerical tests (NNLS, simplex- and box-constrained least squares, orthogonality) demonstrating stability, accuracy, and runtime efficiency of the implicit schemes.