Non-reversible, tuning- and rejection-free Markov chain Monte Carlo via iterated random functions

Abstract: In this work we present a non-reversible, tuning- and rejection-free Markov chain Monte Carlo which naturally fits in the framework of hit-and-run. The sampler only requires access to the gradient of the log-density function, hence the normalizing constant is not needed. We prove the proposed Markov chain is invariant for the target distribution and illustrate its applicability through a wide range of examples. We show that the sampler introduced in the present paper is intimately related to the continuous sampler of Peters and de With (2012), Bouchard-Cote et al. (2017). In particular, the computation is quite similar in the sense that both are centered around simulating an inhomogenuous Poisson process. The computation can be simplified when the gradient of the log-density admits a computationally efficient directional decomposition into a sum of two monotone functions. We apply our sampler in selective inference, gaining significant improvement over the formerly used sampler (Tian et al. 2016).