Properties and applications of Fisher distribution on the rotation group

Abstract: We study properties of Fisher distribution (von Mises-Fisher distribution, matrix Langevin distribution) on the rotation group SO(3). In particular we apply the holonomic gradient descent, introduced by Nakayama et al. (2011), and a method of series expansion for evaluating the normalizing constant of the distribution and for computing the maximum likelihood estimate. The rotation group can be identified with the Stiefel manifold of two orthonormal vectors. Therefore from the viewpoint of statistical modeling, it is of interest to compare Fisher distributions on these manifolds. We illustrate the difference with an example of near-earth objects data.