Adapting Random-X optimism to tree-based boosting ensembles

Develop an optimism framework for tree-based boosting models under the Random-X setting, particularly for boosting ensembles such as gradient boosting and Bayesian additive regression trees, by deriving tractable expressions or estimators for the expected training–testing discrepancy (optimism) that enable prediction-oriented model selection beyond bagging-style ensembles.

Background

The paper establishes closed-form and plug-in characterizations of Random-X optimism for tensor regression with CP and Tucker decompositions and shows that optimism is minimized at the true rank. It further analyzes ensemble settings in a bagging-style context for CP regression and provides upper bounds for ensemble optimism.

In contrast, extending optimism analysis to boosting-based tree ensembles is not addressed. The authors explicitly highlight that adapting the Random-X optimism framework to tree-based boosting remains unresolved, identifying it as a significant open problem in need of theoretical development.