Riemannian Proximal Sampler for High-accuracy Sampling on Manifolds

Abstract: We introduce the Riemannian Proximal Sampler, a method for sampling from densities defined on Riemannian manifolds. The performance of this sampler critically depends on two key oracles: the Manifold Brownian Increments (MBI) oracle and the Riemannian Heat-kernel (RHK) oracle. We establish high-accuracy sampling guarantees for the Riemannian Proximal Sampler, showing that generating samples with $\varepsilon$-accuracy requires $O(\log(1/\varepsilon))$ iterations in Kullback-Leibler divergence assuming access to exact oracles and $O(\log^2(1/\varepsilon))$ iterations in the total variation metric assuming access to sufficiently accurate inexact oracles. Furthermore, we present practical implementations of these oracles by leveraging heat-kernel truncation and Varadhan's asymptotics. In the latter case, we interpret the Riemannian Proximal Sampler as a discretization of the entropy-regularized Riemannian Proximal Point Method on the associated Wasserstein space. We provide preliminary numerical results that illustrate the effectiveness of the proposed methodology.

• The paper presents a novel sampler that guarantees high-accuracy sampling with O(log(1/ε)) iterations using exact oracles.
• It leverages Manifold Brownian Increments and Riemannian Heat-kernel oracles to adapt sampling methods to complex manifold structures.
• Empirical evaluations on hyperspheres and positive definite matrices show notable improvements over traditional Riemannian Langevin Monte Carlo.

Riemannian Proximal Sampler for High-Accuracy Sampling on Manifolds

The paper presented introduces the Riemannian Proximal Sampler, an innovative approach aiming to enhance the efficiency of sampling from densities defined over Riemannian manifolds. This approach is particularly significant in contexts where traditional Euclidean sampling methodologies fall short due to the intrinsic geometric complexities of manifolds. The paper provides both theoretical guarantees and practical methodologies for the deployment of this new sampler.

Methodology

The Riemannian Proximal Sampler relies on two crucial computational components: the Manifold Brownian Increments (MBI) oracle and the Riemannian Heat-kernel (RHK) oracle. The MBI oracle is responsible for generating manifold-constrained samples akin to Brownian motion in Euclidean spaces, while the RHK oracle is tasked with ensuring convergence towards the target distribution by addressing geometric irregularities entailed by manifolds.

The authors guarantee high-accuracy sampling, establishing that generating samples with $\varepsilon$-accuracy requires $\mathcal{O}(\log(1/\varepsilon))$ iterations in Kullback-Leibler divergence, given that exact oracles are available. In situations where inexact oracles are employed, they demonstrate performance bounds of $\mathcal{O}(\log^2(1/\varepsilon))$ iterations in the total variation metric.

Practical Implementations

The paper further elucidates practical implementations of the MBI and RHK oracles through methodologies encompassing heat-kernel truncation and Varadhan's asymptotics. By truncating infinite series and leveraging the asymptotic properties of heat kernels, the authors provide feasible avenues for their approximation under computational constraints.

The implementation of the RHK oracle connects this sampling method with the entropy-regularized proximal point framework, often employed in Wasserstein spaces, thereby reinforcing the applicability and robustness of their scheme in attaining convergence within manifold settings.

Empirical Evaluation

To substantiate their claims, the authors demonstrate the efficacy of the Riemannian Proximal Sampler through empirical evaluation, showcasing significant improvements in scenarios involving hyperspheres and positive definite matrices. These experiments reveal the sampler's potential in overcoming challenges associated with curvature and provide evidence of its capacity to outperform traditional Riemannian Langevin Monte Carlo methods within certain parameter regimes.

Implications and Future Work

The development of the Riemannian Proximal Sampler introduces a promising direction for high-accuracy sampling in machine learning and broader scientific fields where data inherently lie on complex manifolds. The connection drawn between the proximal sampler methodology and the entropy-regularized Wasserstein framework lays the groundwork for future explorations into advanced manifold sampling techniques.

Future work could build upon this to refine the accuracy and efficiency of manifold sampling algorithms. Potential areas for further investigation include extending the applicability to broader classes of manifolds and enhancing the computational efficiency of oracle implementations under varying practical constraints. Moreover, exploring adaptive methods to better accommodate negative curvature spaces could widen the versatility of these sampling techniques.

In conclusion, the Riemannian Proximal Sampler signifies an important contribution to manifold-based statistical methodologies, broadening the toolkit available for statisticians and machine learning practitioners working with non-Euclidean data.