On the error control of invariant causal prediction

Abstract: Invariant causal prediction (ICP, Peters et al. (2016)) provides a novel way for identifying causal predictors of a response by utilizing heterogeneous data from different environments. One notable advantage of ICP is that it guarantees to make no false causal discoveries with high probability. Such a guarantee, however, can be overly conservative in some applications, resulting in few or no causal discoveries. This raises a natural question: Can we use less conservative error control guarantees for ICP so that more causal information can be extracted from data? We address this question in the paper. We focus on two commonly used and more liberal guarantees: false discovery rate control and simultaneous true discovery bound. Unexpectedly, we find that false discovery rate does not seem to be a suitable error criterion for ICP. The simultaneous true discovery bound, on the other hand, proves to be an ideal choice, enabling users to explore potential causal predictors and extract more causal information. Importantly, the additional information comes for free, in the sense that no extra assumptions are required and the discoveries from the original ICP approach are fully retained. We demonstrate the practical utility of our method through simulations and a real dataset about the educational attainment of teenagers in the US.