Partially Pooled Propensity Score Models for Average Treatment Effect Estimation with Multilevel Data

Abstract: Causal inference analyses often use existing observational data, which in many cases has some clustering of individuals. In this paper we discuss propensity score weighting methods in a multilevel setting where within clusters individuals share unmeasured confounders that are related to treatment assignment and the potential outcomes. We focus in particular on settings where models with fixed cluster effects are either not feasible or not useful due to the presence of a large number of small clusters. We found, both through numerical experiments and theoretical derivations, that a strategy of grouping clusters with similar treatment prevalence and estimating propensity scores within such cluster groups is effective in reducing bias from unmeasured cluster-level covariates under mild conditions on the outcome model. We apply our proposed method in evaluating the effectiveness of center-based pre-school program participation on children's achievement at kindergarten, using the Early Childhood Longitudinal Study, Kindergarten data.