Solving Bayesian inverse problems via Fisher adaptive Metropolis adjusted Langevin algorithm

Abstract: The preconditioned Metropolis adjusted Langevin algorithm (MALA) is a widely used method in statistical applications, where the choice of the preconditioning matrix plays a critical role. Recently, Titsias \cite{Titsias2024} demonstrated that the inverse Fisher information matrix is the optimal preconditioner by minimizing the expected squared jump distance and proposed an adaptive scheme to estimate the Fisher matrix using the sampling history. In this paper, we apply the Fisher adaptive Metropolis adjusted Langevin algorithm (MALA) to Bayesian inverse problems. Moreover, we provide a rigorous convergence rate analysis for the adaptive scheme used to estimate the Fisher matrix. To evaluate its performance, we use this algorithm to sample from posterior distributions in several Bayesian inverse problems. And compare its constructions with the standard adaptive Metropolis adjusted Langevin algorithm (which employs the empirical covariance matrix of the posterior distribution as the preconditioner) and the preconditioned Crank-Nicolson (pCN) algorithm. Our numerical results demonstrate show that the Fisher adaptive MALA is highly effective for Bayesian inversion, and significantly outperforms other sampling methods, particularly in high-dimensional settings.