Robustness Metric for Quantifying Causal Model Confidence and Parameter Uncertainty

Abstract: Many methods of estimating causal models do not provide estimates of confidence in the resulting model. In this work, a metric is proposed for validating the output of a causal model fit; the robustness of the model structure with resampled data. The metric is developed for time series causal models, but is also applicable to non-time series data. The proposed metric may be utilized regardless of the method selected for fitting the causal model. We find that with synthetically generated data, this metric is able to successfully identify the true data generating model in most cases. Additionally, the metric provides both a qualitative measure of model confidence represented by the robustness level as well as accurate estimates of uncertainty in model coefficients which are important in interpreting model results. The use of this metric is demonstrated on both numerically simulated data and a case study from existing causal model literature.