On the frequentist validity of Bayesian limits

Abstract: To the frequentist who computes posteriors, not all priors are useful asymptotically: in this paper Schwartz's 1965 Kullback-Leibler condition is generalised to enable frequentist interpretation of convergence of posterior distributions with the complex models and often dependent datasets in present-day statistical applications. We prove four simple and fully general frequentist theorems, for posterior consistency; for posterior rates of convergence; for consistency of the Bayes factor in hypothesis testing or model selection; and a theorem to obtain confidence sets from credible sets. The latter has a significant methodological consequence in frequentist uncertainty quantification: use of a suitable prior allows one to convert credible sets of a calculated, simulated or approximated posterior into asymptotically consistent confidence sets, in full generality. This extends the main inferential implication of the Bernstein-von Mises theorem to non-parametric models without smoothness conditions. Proofs require the existence of a Bayesian type of test sequence and priors giving rise to local prior predictive distributions that satisfy a weakened form of Le~Cam's contiguity with respect to the data distribution. Results are applied in a wide range of examples and counterexamples.