Asymptotic Normality of Generalized Low-Rank Matrix Sensing via Riemannian Geometry

Abstract: We prove an asymptotic normality guarantee for generalized low-rank matrix sensing -- i.e., matrix sensing under a general convex loss $\bar\ell(\langle X,M\rangle,y^*)$, where $M\in\mathbb{R}^{d\times d}$ is the unknown rank-$k$ matrix, $X$ is a measurement matrix, and $y^*$ is the corresponding measurement. Our analysis relies on tools from Riemannian geometry to handle degeneracy of the Hessian of the loss due to rotational symmetry in the parameter space. In particular, we parameterize the manifold of low-rank matrices by $\bar\theta\bar\theta^\top$, where $\bar\theta\in\mathbb{R}^{d\times k}$. Then, assuming the minimizer of the empirical loss $\bar\theta^0\in\mathbb{R}^{d\times k}$ is in a constant size ball around the true parameters $\bar\theta^*$, we prove $\sqrt{n}(\phi^0-\phi^)\xrightarrow{D}N(0,(H^)^{-1})$ as $n\to\infty$, where $\phi^0$ and $\phi^*$ are representations of $\bar\theta^*$ and $\bar\theta^0$ in the horizontal space of the Riemannian quotient manifold $\mathbb{R}^{d\times k}/\text{O}(k)$, and $H^*$ is the Hessian of the true loss in the same representation.