Explainable Boosting Machines (EBMs)

• Explainable Boosting Machines are glass-box models that combine interpretable generalized additive models with cyclic gradient boosting for clear, feature-level insights.
• They employ discretization, shallow tree ensembles, and a round-robin boosting strategy to accurately capture both univariate and pairwise interactions.
• EBMs support global and local interpretability, making them suitable for applications needing transparency in high-dimensional, regulatory, and privacy-focused settings.

Explainable Boosting Machines (EBMs) are a class of glass-box machine learning models that combine the predictive power of modern ensemble techniques with rigorous, exact interpretability. At their core, EBMs are generalized additive models (GAMs) enhanced through cyclic gradient boosting over shallow tree ensembles for both univariate and selected pairwise interaction terms. The construction, training, and post-processing of EBMs are designed to provide a transparent mapping from features to predictions, supporting both global and local interpretability across tabular, scientific, and high-dimensional data, as well as in settings requiring fairness, privacy, and regulatory compliance.

1. Mathematical Foundation and Model Structure

The structural form of an EBM is that of a generalized additive model with optional pairwise interactions (“GA²M”):

$$g\bigl(\mathbb{E}[y \mid \boldsymbol{x}]\bigr) = \theta_0 + \sum_{j=1}^p f_j(x_j) + \sum_{1 \leq j < k \leq p} f_{jk}(x_j, x_k)$$

where:

• $g(\cdot)$ is a link function appropriate for the task (e.g., identity for regression, logit for binary classification, log for count data or positive outcomes),
• $\theta_0$ is the global intercept,
• $f_j$ are univariate “shape” functions, learned as sums of shallow tree ensembles,
• $f_{jk}$ are pairwise interaction functions, optionally included for top-ranked feature pairs.

Prediction for an input $\boldsymbol{x}$ is made by evaluating each $f_j(x_j)$ and relevant $f_{jk}(x_j, x_k)$, then inverting the link function.

This additive and low-dimensional functional decomposition enables direct visualization and attribution of model predictions at both global (across the training domain) and local (per-sample) levels (Greenwell et al., 2023, Schug et al., 2023, Krùpovà et al., 27 Mar 2025).

2. Training via Cyclic Gradient Boosting

EBMs are distinguished by the use of a cyclic (coordinate-wise) gradient boosting algorithm:

• Discretization: All features are discretized into bins—quantiles for continuous data, categorical levels as-is. For interactions, the joint bins are the cross-product grid of constituent univariate bins.
• Cyclic boosting: At each iteration, the algorithm cycles through individual terms (first over main effects, then over selected interactions), fitting a shallow decision tree (typically depth 2–4) to the current pseudo-residuals for that term only.
• Update: The tree’s contributions are added with a small learning rate to update the corresponding function $f_j$ or $f_{jk}$.
• Early stopping: Model growth is halted if out-of-sample deviance does not improve.
• Interaction selection: After univariate terms are fit, EBMs use the GA²M “FAST” procedure to score and select a small set of high-impact interactions, which are then trained analogously (Krùpovà et al., 27 Mar 2025, Nori et al., 2021).

This round-robin approach yields robustness to feature collinearity and rare categorical or long-tail values, and ensures that each term captures distinct, interpretable marginal or interaction effects (Greenwell et al., 2023).

3. Interpretability and Visualization

Because the EBM is a sum of univariate and bivariate components, interpretability is exact and not an approximation or surrogate:

• Global interpretability: Shape functions $f_j$ and surfaces $f_{jk}$ can be plotted to reveal main and interaction effects directly. These curves (or heatmaps) completely describe the model’s logic for each term.
• Local interpretability: For any sample $\boldsymbol{x}$, its prediction is decomposed exactly into additive contributions $f_j(x_j)$, $f_{jk}(x_j, x_k)$, with no post-hoc estimation.
• No hidden weights: Unlike neural networks or black-box ensembles, all model parameters are transparent, plotted, and inspectable (Greenwell et al., 2023, Nori et al., 2021, Krùpovà et al., 27 Mar 2025).

In comparative settings (e.g., car insurance risk modeling), EBM shape functions have matched domain knowledge about U-shaped risk effects, risk stratification by categorial class, and strong feature-product associations observable as interaction heatmaps (Krùpovà et al., 27 Mar 2025).

4. Challenges and Advances in Interpretability

Despite EBMs' glass-box design, several challenges arise in practice:

• Spurious interactions: Pairwise interaction terms can be artificially inflated by redundant or noisy features, defined as “spurious” if one member lies in the lowest 10% of main-effect importance but participates in a top-10%-ranked interaction. This undermines interpretability as modeled dependencies may be unrelated to genuine underlying structure.
• Single-feature dominance: Occurs when a single feature appears disproportionately in the top-K interaction terms (e.g., up to 5 of 5), which may mask the contribution of other relevant features (R et al., 2023).

A multi-stage cross-feature selection approach, comprising feature selection by ensemble of selectors (SHAP, XGBoost, Boruta, etc.), aggregation into robust feature sets, and filtered inclusion of interactions based on main-effect and interaction score thresholds, can substantially reduce both issues. Empirically, this eliminates all bottom-10%-spurious interactions, reduces over-dominant features to at most 2 out of 5, and improves both interpretability and predictive performance (R et al., 2023).

5. Scalability and Sparsity in High Dimensions

With hundreds or thousands of predictors, vanilla EBMs can become cumbersome:

• Transparency loss: Large numbers of main or interaction terms make model explanations unmanageable.
• Scoring latency: Each term incurs marginal scoring cost, increasing prediction time and resource usage.

LASSO-based post-processing can induce sparsity by reweighting and removing EBM terms with nonnegative or group constraints. A fitted EBM is viewed as a linear model over its learned terms; LASSO is then applied to this basis, zeroing out low-importance effects. In benchmark datasets, this reduces terms by 80–95% without material accuracy loss (e.g., from 369+10 to 19 nonzero terms, with minimal MSE degradation), enabling faster and more transparent deployment (Greenwell et al., 2023).

6. Extensions: Scientific Images, Fairness, and Privacy

EBMs have proven adaptable beyond classical tabular settings:

• Scientific image data: By extracting structured feature summaries from images (e.g., Gabor wavelet transforms pooled by region/quadrant), EBMs can model high-fidelity, interpretable relationships in physical science and quantum imaging. Empirical results in cold-atom soliton classification show EBM-based models achieving accuracy on par with deep learning alternatives and better alignment between learned effects and physicist intuition (Schug et al., 2023).
• Differential privacy: Adding calibrated Gaussian noise at the residual-aggregation step in boosting enables (ε, δ)-differentially private training with negligible accuracy loss. Unlike other private models, EBM post-processing and monotonicity-enforcement impose no additional privacy cost thanks to DP’s post-processing property (Nori et al., 2021).
• Fair and regulatory applications: EBMs’ exact breakdown of predictions into human-comprehensible effects supports their use in regulated industries where audit trails, transparency, and feature-level dependency checks are required (e.g., lending, insurance) (R et al., 2023, Krùpovà et al., 27 Mar 2025).

7. Quantitative Benchmarks and Practical Considerations

Quantitative benchmarking on public datasets indicates that EBMs:

• Match or exceed classical methods (GLMs, spline GAMs, pruned CARTs) in RMSE, explained deviance, and AUROC,
• Approach the accuracy of black-box models (XGBoost) when a modest number of interaction terms is included,
• Yield substantial improvements in feature selection stability and model scoring efficiency via cross-feature selection and LASSO post-processing,
• Provide fully transparent, glass-box explanations at both global and local levels, eliminating the need for post-hoc attribution such as SHAP or LIME (Krùpovà et al., 27 Mar 2025, R et al., 2023, Greenwell et al., 2023).

In summary, Explainable Boosting Machines represent a state-of-the-art approach balancing accuracy, scalability, and genuine interpretability, underpinned by precise mathematical structure and robust algorithmic design across a range of modern learning settings (Greenwell et al., 2023, Nori et al., 2021, Krùpovà et al., 27 Mar 2025, R et al., 2023, Schug et al., 2023).