Infinite dimensional adaptive MCMC for Gaussian processes

Abstract: Latent Gaussian processes are widely applied in many fields like, statistics, inverse problems and machine learning. A popular method for inference is through the posterior distribution, which is typically carried out by Markov Chain Monte Carlo (MCMC) algorithms. Most Gaussian processes can be represented as a Gaussian measure in a infinite dimensional space. This is an issue for standard algorithms as they break down in an infinite dimensional setting, thus the need for appropriate infinite dimensional samplers for implementing probabilistic inference in such framework. In this paper, we introduce several adaptive versions of the preconditioned Crank-Nicolson Langevin (pCNL) algorithm, which can be viewed as an infinite dimensional version of the well known Metropolis adjusted Langevin algorithm (MALA) algorithm for Gaussian processes. The basic premise for all our proposals lies in the idea of implementing change of measure formulation to adapt the algorithms to greatly improve their efficiency. A gradient-free version of pCNL is introduced, which is a hybrid of an adaptive independence sampler and an adaptive random walk sampler, and is shown to outperform the standard preconditioned Crank-Nicolson (pCN) scheme. Finally, we demonstrate the efficiency of our proposed algorithm for three different statistical models.