Optimization on the symplectic group

Abstract: We regard the real symplectic group $Sp(2n,\mathbb{R})$ as a constraint submanifold of the $2n\times 2n$ real matrices $\mathcal{M}_{2n}(\mathbb{R})$ endowed with the Euclidean (Frobenius) metric, respectively as a submanifold of the general linear group $Gl(2n,\mathbb{R})$ endowed with the (left) invariant metric. For a cost function that defines an optimization problem on the real symplectic group we give a necessary and sufficient condition for critical points and we apply this condition to the particular case of a least square cost function. In order to characterize the critical points we give a formula for the Hessian of a cost function defined on the real symplectic group, with respect to both considered metrics. For a generalized Brockett cost function we present a necessary condition and a sufficient condition for local minimum. We construct a retraction map that allows us to detail the steepest descent and embedded Newton algorithms for solving an optimization problem on the real symplectic group.