Inference of Sample Complier Average Causal Effects under Experiments with Completely Randomized Design and Computer Assisted Balance-Improving Designs

Abstract: Non-compliance is common in real world experiments. We focus on inference about the sample complier average causal effect, that is, the average treatment effect for experimental units who are compliers. We present three types of inference strategies for the sample complier average causal effect: the Wald estimator, regression adjustment estimators and model-based Bayesian inference. Because modern computer assisted experimental designs have been used to improve covariate balance over complete randomization, we discuss inference under both complete randomization and a specific computer assisted experimental design - Mahalanobis distance based rerandomization, under which asymptotic properties of the Wald estimator and regression adjustment estimators can be derived. We use Monte Carlo simulation to compare the finite sample performance of the methods under both experimental designs. We find that under either design, the Bayesian method performs the best because it is stable, it yields smallest median absolute error and smallest median interval length. The improvement by the Bayesian method is especially large when the fraction of compliers is small. We present an application to a job training experiment with non-compliance.