BeyondPlanck IV. On end-to-end simulations in CMB analysis -- Bayesian versus frequentist statistics

Abstract: End-to-end simulations play a key role in the analysis of any high-sensitivity CMB experiment, providing high-fidelity systematic error propagation capabilities unmatched by any other means. In this paper, we address an important issue regarding such simulations, namely how to define the inputs in terms of sky model and instrument parameters. These may either be taken as a constrained realization derived from the data, or as a random realization independent from the data. We refer to these as Bayesian and frequentist simulations, respectively. We show that the two options lead to significantly different correlation structures, as frequentist simulations, contrary to Bayesian simulations, effectively include cosmic variance, but exclude realization-specific correlations from non-linear degeneracies. Consequently, they quantify fundamentally different types of uncertainties, and we argue that they therefore also have different and complementary scientific uses, even if this dichotomy is not absolute. Before BeyondPlanck, most pipelines have used a mix of constrained and random inputs, and used the same hybrid simulations for all applications, even though the statistical justification for this is not always evident. BeyondPlanck represents the first end-to-end CMB simulation framework that is able to generate both types of simulations, and these new capabilities have brought this topic to the forefront. The Bayesian BeyondPlanck simulations and their uses are described extensively in a suite of companion papers. In this paper we consider one important applications of the corresponding frequentist simulations, namely code validation. That is, we generate a set of 1-year LFI 30 GHz frequentist simulations with known inputs, and use these to validate the core low-level BeyondPlanck algorithms; gain estimation, correlated noise estimation, and mapmaking.