An MCMC hypothesis test to check a claimed sampler: applied to a claimed sampler for the G-Wishart distribution

Abstract: Suppose we have a distribution of interest, with density $p(x),x\in {\cal X}$ say, and an algorithm claimed to generate samples from $p(x)$. Moreover, assume we have available a Metropolis--Hastings transition kernel fulfilling detail balance with respect to $p(x)$. In such a situation we formulate a hypothesis test where $H_0$ is that the claimed sampler really generates correct samples from $p(x)$. We use that if initialising the Metropolis--Hastings algorithm with a sample generated by the claimed sampler and run the chain for a fixed number of updates, the initial and final states are exchangeable if $H_0$ is true. Combining this idea with the permutation strategy we define a natural test statistic and a valid p-value. Our motivation for considering the hypothesis test situation is a proposed sampler in the literature, claimed to generate samples from G-Wishart distribution. As no proper proof for the validity of this sampler seems to be available, we are exactly in the hypothesis test situation discussed above. We therefore apply the defined hypothesis test to the claimed sampler. For comparison we also apply the hypothesis test to a known exact sampler for a subset of G-Wishart distributions. The obtained p-values clearly show that the sampler claimed to be able to generate samples from any G-Wishart distribution is in fact not sampling from the specified distribution. In contrast, and as one should expect, the p-values obtained when using the known exact algorithm does not indicate any problems.