Experimentation for Homogenous Policy Change

Abstract: When the Stable Unit Treatment Value Assumption is violated and there is interference among units, there is not a uniquely defined Average Treatment Effect, and alternative estimands may be of interest. Among these are average unit-level differences in outcomes under different homogeneous treatment policies. We refer to such targets as Global Average Treatment Effects. We consider approaches to experimental design with multiple treatment conditions under partial interference and, given the estimand of interest, we show that difference-in-means estimators may perform better than correctly specified regression models in finite samples on root mean squared error for such targets. With errors correlated at the cluster level, we demonstrate that two-stage randomization procedures with intra-cluster correlation of treatment strictly between zero and one may dominate one-stage randomization designs on the same metric. Simulations illustrate performance of this approach; we consider an application to online experiments at Facebook.