Papers
Topics
Authors
Recent
Search
2000 character limit reached

Targeted Learning for Variable Importance

Published 4 Nov 2024 in stat.ML, cs.LG, and stat.ME | (2411.02221v1)

Abstract: Variable importance is one of the most widely used measures for interpreting machine learning with significant interest from both statistics and machine learning communities. Recently, increasing attention has been directed toward uncertainty quantification in these metrics. Current approaches largely rely on one-step procedures, which, while asymptotically efficient, can present higher sensitivity and instability in finite sample settings. To address these limitations, we propose a novel method by employing the targeted learning (TL) framework, designed to enhance robustness in inference for variable importance metrics. Our approach is particularly suited for conditional permutation variable importance. We show that it (i) retains the asymptotic efficiency of traditional methods, (ii) maintains comparable computational complexity, and (iii) delivers improved accuracy, especially in finite sample contexts. We further support these findings with numerical experiments that illustrate the practical advantages of our method and validate the theoretical results.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (42)
  1. David Benkeser and Mark van der Laan. The highly adaptive lasso estimator. In 2016 IEEE international conference on data science and advanced analytics (DSAA), pages 689–696. IEEE, 2016.
  2. Efficient and adaptive estimation for semiparametric models, volume 4. Springer, 1993.
  3. Simon H Bickler. Machine learning arrives in archaeology. Advances in Archaeological Practice, 9(2):186–191, 2021.
  4. Monte carlo approximation of bootstrap variances. The American Statistician, 52(4):354–357, 1998.
  5. Leo Breiman. Random forests. Machine learning, 45:5–32, 2001.
  6. Weixin Cai and Mark J van der Laan. One-step targeted maximum likelihood estimation for time-to-event outcomes. Biometrics, 76(3):722–733, 2020.
  7. Philip E Cheng. Strong consistency of nearest neighbor regression function estimators. Journal of Multivariate Analysis, 15(1):63–72, 1984.
  8. Double/debiased machine learning for treatment and structural parameters. The Econometrics Journal, 21(1):C1–C68, 01 2018.
  9. Kernel debiased plug-in estimation: Simultaneous, automated debiasing without influence functions for many target parameters. In Forty-first International Conference on Machine Learning, 2023.
  10. Bootstrap confidence intervals. Statistical science, 11(3):189–228, 1996.
  11. The rashomon importance distribution: Getting rid of unstable, single model-based variable importance. Advances in Neural Information Processing Systems, 36:6267–6279, 2023.
  12. Deep neural networks for estimation and inference: application to causal effects and other semiparametric estimands. arXiv preprint arXiv:1809.09953, 20, 2018.
  13. All models are wrong, but many are useful: Learning a variable’s importance by studying an entire class of prediction models simultaneously. Journal of Machine Learning Research, 20(177):1–81, 2019.
  14. Scientific inference with interpretable machine learning: Analyzing models to learn about real-world phenomena. Minds and Machines, 34(3):32, 2024.
  15. Model-agnostic confidence intervals for feature importance: A fast and powerful approach using minipatch ensembles. arXiv preprint arXiv:2206.02088, 2022.
  16. Ulrike Grömping. Estimators of relative importance in linear regression based on variance decomposition. The American Statistician, 61(2):139–147, 2007.
  17. Demystifying statistical learning based on efficient influence functions. The American Statistician, 76(3):292–304, 2022.
  18. Machine learning and the future of realism. arXiv preprint arXiv:1704.04688, 2017.
  19. Unrestricted permutation forces extrapolation: variable importance requires at least one more model, or there is no free variable importance. Statistics and Computing, 31:1–16, 2021.
  20. Edward H Kennedy. Semiparametric doubly robust targeted double machine learning: a review. arXiv preprint arXiv:2203.06469, 2022.
  21. Distribution-free predictive inference for regression. Journal of the American Statistical Association, 113(523):1094–1111, 2018.
  22. A unified framework for random forest prediction error estimation. Journal of Machine Learning Research, 22(8):1–41, 2021.
  23. Using machine learning to predict urban canopy flows for land surface modeling. Geophysical Research Letters, 50(1):e2022GL102313, 2023.
  24. Alexander R Luedtke and Mark J van der Laan. Optimal targeted learning: confidence intervals for a median parameter. Statistical Methods in Medical Research, 25(3):897–917, 2016.
  25. High-dimensional l⁢_⁢2𝑙_2l\_2italic_l _ 2 boosting: Rate of convergence. arXiv preprint arXiv:1602.08927, 2016.
  26. Machine learning for sociology. Annual Review of Sociology, 45(1):27–45, 2019.
  27. Christoph Molnar. Interpretable machine learning. Lulu. com, 2020.
  28. Relating the partial dependence plot and permutation feature importance to the data generating process. In World Conference on Explainable Artificial Intelligence, pages 456–479. Springer, 2023.
  29. Cross-fitting and fast remainder rates for semiparametric estimation. arXiv preprint arXiv:1801.09138, 2018.
  30. A general theory of hypothesis tests and confidence regions for sparse high dimensional models. The Annals of Statistics, 2017.
  31. Conditional variable importance for random forests. BMC bioinformatics, 9:1–11, 2008.
  32. Mark J van der Laan. Statistical inference for variable importance. The International Journal of Biostatistics, 2(1), 2006.
  33. Mark J van der Laan and Daniel Rubin. Targeted maximum likelihood learning. The international journal of biostatistics, 2(1), 2006.
  34. Cross-validated targeted minimum-loss-based estimation. Targeted learning: causal inference for observational and experimental data, pages 459–474, 2011a.
  35. Targeted learning: causal inference for observational and experimental data, volume 4. Springer, 2011b.
  36. Aad W van der Vaart. Asymptotic statistics, volume 3. Cambridge university press, 2000.
  37. Decorrelated variable importance. Journal of Machine Learning Research, 25(7):1–27, 2024.
  38. Efficient targeted learning of heterogeneous treatment effects for multiple subgroups. Biometrics, 79(3):1934–1946, 2023.
  39. Efficient nonparametric statistical inference on population feature importance using shapley values. In International conference on machine learning, pages 10282–10291. PMLR, 2020.
  40. Nonparametric variable importance assessment using machine learning techniques. Biometrics, 77(1):9–22, 2021.
  41. A general framework for inference on algorithm-agnostic variable importance. Journal of the American Statistical Association, 118(543):1645–1658, 2023.
  42. Nonparametric variable importance for time-to-event outcomes with application to prediction of hiv infection. arXiv preprint arXiv:2311.12726, 2023.

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up

Tweets

Sign up for free to view the 1 tweet with 2 likes about this paper.

Sign Up for Free