A Riemannian optimization method on the indefinite Stiefel manifold

Abstract: We consider the optimization problem with a generally quadratic matrix constraint of the form $X^TAX = J$, where $A$ is a given nonsingular, symmetric $n\times n$ matrix and $J$ is a given $k\times k$ matrix, with $k\leq n$, satisfying $J^2 = I_k$. Since the feasible set constitutes a differentiable manifold, called the indefinite Stiefel manifold, we approach this problem within the framework of Riemannian optimization. Namely, we first equip the manifold with a Riemannian metric and construct the associated geometric structures, then propose a retraction based on the Cayley transform, and finally suggest a Riemannian gradient descent method using the attained materials, whose global convergence is guaranteed. Our results not only cover the known cases, the orthogonal and generalized Stiefel manifolds, but also provide a Riemannian optimization solution for other constrained problems which has not been investigated. Numerical experiments are presented to justify the theoretical results.