Propensity Score Methods for Merging Observational and Experimental Datasets

Abstract: This project considers how one might augment a limited amount of data from randomized controlled trial (RCT) with more plentiful data from an observational database (ODB), in order to estimate a causal effect. In our motivating setting, the ODB has better external validity, while the RCT has genuine randomization. We work with strata defined by the propensity score in the ODB. Subjects from the RCT are placed in strata defined by the propensity they would have had, had they been in the ODB. Our first method simply spikes the RCT data into their corresponding ODB strata. Our second method takes a data-driven convex combination of the ODB and RCT treatment effect estimates within each stratum. Using the delta method and simulations we show that the spike-in method works best when the RCT covariates are drawn from the same distribution as in the ODB. Our convex combination method is more robust than the spike-in to covariate-based inclusion criteria that bias the RCT data. We apply our methods to data from the Women's Health Initiative, a study of thousands of postmenopausal women which has both observational and experimental data on hormone therapy (HT). Using half of the RCT to define a gold standard, we find that a version of the spiked-in estimate yields stable estimates of the causal impact of HT on coronary heart disease.