Asymptotic behavior of standard gradient boosting algorithms

Determine the asymptotic behavior of gradient boosting algorithms commonly used in practice, including Explainable Boosting Machines (EBMs). The goal is to characterize the large-sample limits of these algorithms (e.g., to which estimators they converge and under what conditions) in settings beyond the specific randomized or Boulevard-regularized variants for which asymptotic analyses currently exist.

Background

The paper studies statistical inference for Explainable Boosting Machines (EBMs), an additive tree-based gradient boosting method. While the authors establish asymptotic results for specific Boulevard-regularized EBM variants (including parallel and backfitting-like procedures), they note that, in general, the asymptotic behavior of most gradient boosting algorithms remains unknown.

Prior work has analyzed asymptotics for certain modified boosting schemes: a randomized boosting procedure converging to a Gaussian process and Boulevard regularization converging to kernel ridge regression. However, these do not cover the standard gradient boosting updates widely used in practice. Hence, understanding the general asymptotic behavior for typical gradient boosting algorithms, including EBMs, is an explicit open question.