Translating Machine Learning Interpretability into Clinical Insights for ICU Mortality Prediction

Abstract: Current research efforts largely focus on employing at most one interpretable method to elucidate ML model performance. However, significant barriers remain in translating these interpretability techniques into actionable insights for clinicians, notably due to complexities such as variability across clinical settings and the Rashomon effect. In this study, we developed and rigorously evaluated two ML models along with interpretation mechanisms, utilizing data from 131,051 ICU admissions across 208 hospitals in the United States, sourced from the eICU Collaborative Research Database. We examined two datasets: one with imputed missing values (130,810 patients, 5.58% ICU mortality) and another excluding patients with missing data (5,661 patients, 23.65% ICU mortality). The random forest (RF) model demonstrated an AUROC of 0.912 with the first dataset and 0.839 with the second dataset, while the XGBoost model achieved an AUROC of 0.924 with the first dataset and 0.834 with the second dataset. Consistently identified predictors of ICU mortality across datasets, cross-validation folds, models, and explanation mechanisms included lactate levels, arterial pH, body temperature, and others. By aligning with routinely collected clinical variables, this study aims to enhance ML model interpretability for clinical use, promote greater understanding and adoption among clinicians, and ultimately contribute to improved patient outcomes.