Bregman–Riesz Unified Approaches

• Bregman–Riesz unified approaches are a comprehensive framework that integrates Riesz representer estimation, Bregman divergence minimization, and semiparametric efficiency for debiased inference.
• They leverage convex optimization and duality theory to connect classical methods like Riesz regression, entropy balancing, and TMLE under a unified theoretical foundation.
• They facilitate algorithmic debiasing of plug-in estimators with rigorous guarantees on bias reduction, convergence, and efficiency in high-dimensional settings.

Bregman–Riesz unified approaches constitute a comprehensive statistical and algorithmic framework that integrates the estimation of Riesz representers, Bregman divergence minimization, and semiparametric efficiency theory. This paradigm encompasses Riesz regression, covariate balancing, density-ratio estimation, targeted maximum likelihood estimation (TMLE), entropy balancing, and nearest-neighbor matching as special cases, providing a single theoretical and practical foundation for debiased machine learning and causal inference. Central features include the use of Bregman divergences to measure and control errors in Riesz representer estimation, convex (or strongly convex) optimization to ensure stability and convergence, and duality theory linking primal loss minimization to moment-matching constraints often interpreted as balancing or weighting. This approach enables automatic debiasing of plug-in estimators, robustifies against first-stage bias, and facilitates algorithmic automation for a wide spectrum of structural and causal targets (Kato, 27 Oct 2025, Hines et al., 17 Oct 2025, Kato, 19 Feb 2026, Kato, 12 Jan 2026, Kato, 30 Oct 2025).

1. Theoretical Foundations: Bregman Divergences and Riesz Representers

The unifying element is the marriage of Bregman divergence $D_{\phi}(p\|\ q)$—for strictly convex, differentiable $\phi:\mathbb{R}\to\mathbb{R}$—

$$D_\phi(p\|q) = \phi(p) - \phi(q) - \phi'(q) \cdot (p - q)$$

and the Riesz representation theorem, which, for any bounded linear functional $\ell$ on $L_2(P_X)$, guarantees $\ell(\gamma) = \mathbb{E}[\alpha_0(X)\gamma(X)]$ for a unique $\alpha_0$ (the Riesz representer). For statistical or causal functionals $\theta_0 = \mathbb{E}[m(W, \gamma_0)]$ linear in the regression function $\gamma_0$, the orthogonal score is

$$\psi(W;\gamma,\alpha,\theta) = m(W, \gamma) + \alpha(X)[Y - \gamma(X)] - \theta$$

with Neyman-orthogonality ensuring that plug-in estimators are debiased at first order (Kato, 27 Oct 2025, Kato, 30 Oct 2025). The estimation goal is: given data $\{(X_i, Y_i)\}$, estimate $\alpha_0$ by minimizing the expected Bregman risk.

2. Generalized Riesz Regression: Unified Primal and Dual Problems

Generalized Riesz regression seeks an estimator

$$\alpha^* = \underset{\alpha\in\mathcal{A}}{\arg\min}~ \mathbb{E}\big[ D_\phi(\alpha_0(X) \| \alpha(X)) \big]$$

which, by exploiting the linearity of $m$ and the Riesz representation, reduces (up to additive constants) to minimizing the empirical Bregman–Riesz objective

$$\frac{1}{n}\sum_{i=1}^n~ -\phi(\alpha(X_i)) + \phi'(\alpha(X_i))\alpha(X_i) - m(W_i, \phi'(\alpha(\cdot)))$$

plus penalty $\lambda J(\alpha)$. Key choices of $\phi$ induce standard procedures:

• Squared loss $\phi(u) = \frac{1}{2}u^2$: yields classic Riesz regression or least-squares importance fitting (LSIF) (Hines et al., 17 Oct 2025, Kato, 27 Oct 2025, Kato, 30 Oct 2025, Kato, 12 Jan 2026)
• Kullback–Leibler $\phi(u) = u\log u - u$: yields entropy balancing/covariate balancing (Kato, 27 Oct 2025, Kato, 30 Oct 2025, Kato, 12 Jan 2026)

The Fenchel–Legendre dual yields balancing weights $w_i$ and moment-matching constraints: $$\min_{w} \sum_i \phi(w_i) \quad \text{s.t.} \quad \left| \frac{1}{n} \sum_{i} w_i \phi_j(X_i) - \frac{1}{n} \sum_{i} m(W_i, \phi_j) \right| \leq \lambda$$ with $\phi(t)=t^2/2$ or $t\log t$ yielding stable balancing and entropy balancing weights, respectively (Kato, 12 Jan 2026, Kato, 30 Oct 2025).

3. Integration of Classical and Modern Methods

The Bregman–Riesz framework unifies a wide spectrum of balancing and debiasing methodologies as special cases:

• Riesz regression: Squared loss in the population or empirical Bregman minimization (Kato, 27 Oct 2025, Kato, 30 Oct 2025)
• Covariate balancing: Dual of Bregman Riesz minimization with KL generator corresponds to entropy balancing weights for exact moment-matching in a chosen basis (Kato, 12 Jan 2026, Kato, 30 Oct 2025)
• Targeted Maximum Likelihood Estimation (TMLE): The “fluctuation step” for $\gamma$ using clever covariate $\alpha$ implements the TMLE update (Kato, 27 Oct 2025, Kato, 30 Oct 2025, Kato, 19 Feb 2026)
• Density-ratio estimation: Primal minimization with $m(W,\gamma)=\gamma(X)$ and Riesz representer $\alpha_0(x) = p_1(x)/p_0(x)$ subsumes LSIF (squared loss) and KLIEP (KL loss) (Hines et al., 17 Oct 2025, Kato, 12 Jan 2026)

Nearest-neighbor matching, causal forests, and score-matching for diffusion models are specific parameterizations or choices of basis functions/bregman generators within this framework (Kato, 30 Oct 2025, Kato, 12 Jan 2026).

Method	Bregman Generator ($\phi$)	Dual Interpretation
Riesz regression	$u^2/2$ (squared loss)	Stable balancing weights
Entropy balancing	$u\log u - u$ (KL)	Entropy balancing
LSIF	$u^2/2$	L2 minimization
KLIEP	$u\log u - u$	Max-entropy weights

4. Automated Debiasing and Cross-fitting Algorithms

The direct debiased machine learning (DDML) algorithm alternates between fitting the regression function $\gamma$ and the Riesz representer $\alpha$ using cross-fitting and empirical Bregman divergence minimization:

1. Split data, fit $\gamma$ and $\alpha$ in alternation using designated losses, swap splits and repeat.
2. Aggregate cross-fitted nuisance estimates to construct final plug-in and doubly robust estimators (RA, RW, ARW, TMLE).
3. Main steps remain convex optimization; regularization (e.g., RKHS norm, $\ell_1/\ell_2$ penalty) ensures stability in high-dimensional or nonparametric settings (Kato, 27 Oct 2025, Kato, 19 Feb 2026).

Cross-fitting and Neyman orthogonality of the score ensure that only second-order bias persists, so asymptotic normality and double robustness are retained provided $\|\widehat{\gamma}-\gamma_0\| = o_p(n^{-1/4})$, $\|\widehat{\alpha}-\alpha_0\| = o_p(n^{-1/4})$ (Kato, 27 Oct 2025, Kato, 19 Feb 2026, Kato, 12 Jan 2026).

5. Algorithmic and Software Ecosystem

The genriesz Python package implements generalized Riesz regression with user interfaces for:

• Specifying the target functional $m(W, \cdot)$ as a black-box oracle
• Flexible representer modeling (polynomials, RKHS, neural embeddings, forests, nearest-neighbors)
• Choice of Bregman generator and matching link function (“automatic regressor balancing” ensures dual KKT conditions match moment-matching)
• Output of RA, RW, ARW, TMLE estimators, standard errors, confidence intervals, and $p$-values (Kato, 19 Feb 2026)

Learning density ratios for counterfactual or unobserved distributions leverages data augmentation (e.g., permutation, derangement, synthetic pairing) to generate suitable training samples, applied across causal estimands such as ATE, ATT, and AME (Hines et al., 17 Oct 2025, Kato, 19 Feb 2026).

6. Statistical Guarantees and Empirical Insights

Convergence rates for Bregman–Riesz estimators hold under RKHS or neural-network parameterizations, with minimax rates dictated by the RKHS entropy exponent or the neural network’s pseudo-dimension:

• RKHS: $$\|\widehat{\alpha} - \alpha^*\|_{L_2(P)}^2 = O_P(\sqrt{\lambda})$$ with $\lambda \sim n^{-2/(2+\gamma)}$
• Neural network: $$\|\widehat{\alpha} - \alpha^*\|_{L_2(P)}^2 = O_P(Pdim \log n / n)$$ (Kato, 12 Jan 2026).

ARW and TMLE estimators are asymptotically linear and efficient if cross-fitted nuisances satisfy the mixed-rate condition $$\|\widehat{\alpha}-\alpha_0\|_2 \cdot \|\widehat{\gamma}-\gamma_0\|_2 = o_p(n^{-1/2})$$ (Kato, 27 Oct 2025, Kato, 19 Feb 2026, Kato, 12 Jan 2026, Kato, 30 Oct 2025).

Simulations show that the Bregman divergence choice strongly affects tail control on estimated ratios; negative-binomial and Itakura–Saito divergences can outperform least-squares in low-overlap or high-dimensional settings. Model flexibility (e.g., deep networks) can improve density ratio estimation when properly regularized, especially for complex causal targets (Hines et al., 17 Oct 2025).

7. Implications and Scope of Unified Bregman–Riesz Framework

The Bregman–Riesz unified approaches provide a universal convex-analytic and statistical machinery for constructing debiased, semiparametrically efficient estimators for a wide variety of linear functionals, including average treatment effects, average marginal effects, and counterfactual (density shifted) estimands. Nearly all balancing, weighting, matching, and ratio estimation approaches can be viewed as special cases of population Bregman divergence minimization between true and modelled Riesz representers.

This suggests that new estimators can be engineered by hybridizing Bregman generators or basis functions, and that theoretical guarantees on bias, variance, efficiency, and robustness follow automatically via Neyman orthogonality once α is estimated at sufficient rate. All practical and theoretical advances for one case (e.g., entropy balancing, LSIF, TMLE) propagate across the unified framework (Kato, 27 Oct 2025, Hines et al., 17 Oct 2025, Kato, 12 Jan 2026, Kato, 30 Oct 2025, Kato, 19 Feb 2026).