An Evaluation of Bounding Approaches for Generalization

Abstract: Statisticians have recently developed propensity score methods to improve generalizations from randomized experiments that do not employ random sampling. However, these methods typically rely on assumptions whose plausibility may be questionable in practice. In this article, we introduce and discuss bounding, an approach that is based on alternative assumptions that may be more plausible in a given study. The bounding framework nonparametrically estimates population parameters using a range of plausible values that are consistent with the observed characteristics of the data. We illustrate how the bounds can be tightened using three approaches: imposing an alternative assumption based on monotonicity, redefining the population of inference, and using propensity score stratification. Using the results from two simulation studies, we examine the conditions under which bounds for the population parameter are tightened. We conclude with an application of bounding to SimCalc, a cluster randomized trial that evaluated the effectiveness of a technology aid on mathematics achievement.