Boulevard Regularization: Boosting & Imaging

• Boulevard Regularization is a dual framework that uses a stochastic boosting scheme for regression trees and variational decomposition models to detect elongated, boulevard-like structures.
• The boosting method employs subsampling and modified shrinkage-averaging updates to ensure convergence, reduce overfitting, and facilitate explicit uncertainty quantification.
• In imaging, the approach leverages energy functionals with BV and G-norm penalties to prioritize long, thin features, improving the accuracy of road or ribbon detections.

Boulevard regularization refers to two distinct frameworks in the literature: (1) the Boulevard regularization scheme for gradient-boosted regression trees, and (2) regularization strategies for the enhancement and detection of boulevard-like structures (long, thin features such as roads) in images via variational decomposition models. Both exploit domain-specific regularization, either through ensemble learning design or by variational energy functionals, to attain favorable statistical or structural properties.

1. Boulevard Regularization in Boosted Trees

Boulevard is a regularized stochastic gradient boosting method targeting regression, introducing two principal mechanisms: stochastic subsampling and a modified shrinkage-averaging update. The goal is to ensure convergence of the boosting trajectory and to facilitate explicit uncertainty quantification for predictions (Zhou et al., 2018).

Subsampling and Averaging

At each iteration $b$ of boosting, a random subsample $w \subset \{1,\dots,n\}$ of size $\lfloor \theta n \rfloor$ ($\theta \in (0,1]$) is drawn, and the tree $t_b$ is fitted to residuals on $w$. This stochastic step reduces correlation among trees and attenuates overfitting. The update departs from standard additive schemes: $$f_b(x) = \frac{b-1}{b} f_{b-1}(x) + \frac{\lambda}{b} t_b(x)$$ with $\lambda \in (0,1]$ the learning rate. By telescoping,

$f_b(x) = \frac{\lambda}{b} \sum_{i=1}^b t_i(x),$

so that each ensemble member’s weight decays with $b$. Final predictions are rescaled by $(1+\lambda)/\lambda$ to counteract persistent shrinkage.

Boulevard Algorithmic Framework

The typical Boulevard algorithm follows these steps:

1. Initialize $\hat f_0(x) = 0$.
2. For $b = 1, \dots, B$:
   • Compute current residuals $z_i = y_i - \hat f_{b-1}(x_i)$.
   • Draw a random subsample $w$ (or use the full set if $\theta=1$).
   • Train a regression tree $t_b$ on $\{(x_i, z_i): i \in w\}$.
   • Update ensemble: $$\hat f_b(x) = \frac{b-1}{b}\hat f_{b-1}(x) + \frac{\lambda}{b} t_b(x)$$.
3. Output $\hat f(x) = \frac{1+\lambda}{\lambda} \hat f_B(x)$.

2. Limiting Distribution and Statistical Guarantees

Boulevard’s regularization admits rigorous limiting behavior under technical conditions: structure–value isolation (independence of partitioning from value assignment) and non-adaptivity (structure distribution does not change over boosting progress) (Zhou et al., 2018).

Finite-Sample Convergence

Analytically, as $B \rightarrow \infty$ (number of trees), Boulevard’s fitted vector $(\hat f_b(x_1), \ldots, \hat f_b(x_n))$ converges to the solution of

$$Y^* = \left( \frac{1}{\lambda} I + K_n \right)^{-1} K_n Y,$$

where $K_n$ is the expectation of the “structure matrix” encoding tree partitionings over the training set.

Asymptotic Normality

As $n \to \infty$, under additional tree-shrinking assumptions, for any test point $x$,

$$\frac{\hat f_n(x) - \frac{\lambda}{1+\lambda} f(x)}{r_n} \xrightarrow{d} N(0, \sigma_\epsilon^2)$$

with $r_n$ a variance scaling term computable from the ensemble, and $\sigma_\epsilon^2$ the noise variance. The bias induced by shrinkage vanishes after the prescribed final rescaling.

3. Uncertainty Quantification via Boulevard

Boulevard’s limiting Gaussianity of predictions enables explicit analytic uncertainty intervals (termed “reproduction intervals”): $$\hat f_n(x) \pm z_{1-\alpha/2} \hat \sigma_\epsilon r_n,$$ where $r_n$ and $\hat\sigma_\epsilon$ are estimated from the ensemble and residual variance. This contrasts with conventional gradient boosting—where such analytic intervals are not readily available—and supports calibrated predictive uncertainty, with simulation studies affirming near-nominal coverage (Zhou et al., 2018).

4. Empirical Performance and Predictive Behavior

On both synthetic and real regression tasks (e.g., Boston housing, protein structure datasets), Boulevard matches the mean-squared error of Random Forest and classical gradient boosting machines, without requiring early stopping. The boosting path is more stable due to averaging/shrinkage, and reproduction intervals empirically exhibit coverage rates close to the theoretical 95%. Repeated fits confirm the predicted asymptotic Gaussianity of $\hat f_n(x)$, and ensemble limits can be observed converging toward the kernel ridge form predicted by the theory.

5. Boulevard Regularization in Penalized Image Decomposition

A distinct usage of boulevard-related regularization arises in the detection of long, thin objects (“boulevards”) such as roads, within the BV–G (bounded variation plus Meyer–G) variational image decomposition model (Gilles et al., 2024). Here, boulevard regularization refers to the minimization: $$E(u,v,w) = \|u\|_{BV} + \lambda \|v\|_{L^2}^2 + \mu \|w\|_G \ \text{subject to} \quad f = u + v + w,$$ where $u$ models piecewise-smooth structure, $v$ models small-scale noise, and $w$ (in Meyer’s $G$ space) captures oscillatory “texture.”

BV–G Theorem and Boulevard Enhancement

Theoretical analysis shows that for long-thin objects of width $\epsilon$, their $G$-norm is $O(\epsilon)$, and $BV$-norm is $O(L)$ for length $L \gg 1$. With parameters selected such that $\epsilon < \sqrt{\pi/(\lambda \mu)}$ and $\mu < 4\lambda(L+\epsilon)$, these elongated features are energetically favored in $w$. Therefore boulevard-like objects (roads, ribbons) are enhanced in the “texture” component after decomposition.

Numerical and Application Pipeline

Optimizing $E$ is performed by an alternating scheme using Chambolle’s projector for $G$-norm and $BV$ penalties. The $w$ component is then processed by a line-segment detector and refined by active contour modeling to extract connected road/avenue structures in overhead imagery. This produces high-contrast, accurate boulevard/road detections, with empirical studies reporting improvement in detection precision and recall relative to edge-based strategies (Gilles et al., 2024).

6. Interpretations, Limitations, and Connections

Boulevard regularization, as formulated for boosting, structurally shifts the ensemble predictor toward averaged, kernel-like limits, thus providing both statistical (asymptotic normality, explicit variance) and practical (stability, overfitting resistance) advantages under appropriate conditions. In image processing, its conceptual counterpart, via energy penalization, dictates which image geometries emerge in each variational component. Both perspectives rest on the principle that informed regularization—through stochasticity or variational priors—aligns the estimation process with desirable analytic properties.

A plausible implication is that similar regularization strategies (shrinkage with stochastic structure) may yield analogous convergence and uncertainty quantification properties in other ensemble or variational settings, provided the technical assumptions (e.g., isolation, non-adaptivity, convexity) are met. The connection to kernel ridge regression in boosting, and the analytic descriptive power of the $G$-norm in imaging, suggest that boulevard regularization methods can be considered prototypical approaches in the respective fields for combining practical modeling with provable statistical guarantees (Zhou et al., 2018, Gilles et al., 2024).