Minimax Rates in Statistical Estimation

• Minimax rates are optimal risk decay bounds in estimation, quantifying the fastest achievable error rates uniformly over model classes.
• They balance bias and variance through intricate tradeoffs influenced by sample size, function smoothness, and feature dimensionality.
• Applications span nonparametric regression, random forests, inverse problems, and community detection, guiding practical algorithm design.

Minimax rates characterize the fundamental limits for statistical estimation and learning within specified problem classes, quantifying the optimal decay of risk or error as a function of sample size, model parameters, and function class complexity. The minimax rate is the fastest (typically order-optimal) rate achievable by any estimator (or procedure), uniformly over a function or model class, making it central to statistical theory and the design of learning algorithms.

1. Formal Definition and General Principle

The minimax risk for a statistical estimation problem is defined as the infimum over all estimators of the maximum expected loss across the target class. For a parameter space $\Theta$ and loss $\ell$, this is formalized as: $$R_n^* = \inf_{\hat{\theta}} \sup_{\theta \in \Theta} \mathbb E_\theta[\ell(\hat{\theta}, \theta)]$$ For function estimation (e.g., regression or density estimation), the risk may be expressed as squared error, $L_2$ norm, $L_1$ norm, or other metrics depending on the application. The minimax rate is the asymptotic order of $R_n^*$ as $n \to \infty$. The goal is to characterize $R_n^*$, often in terms of intrinsic complexity measures—such as metric entropy, covering numbers, smoothness parameters, and dimensionality.

2. Metric Entropy and the "Le Cam Equation"

Minimax rates in nonparametric estimation are fundamentally determined by the metric entropy structure of the function or model class. The prototypical characterization is via localized metric entropy $H(\varepsilon)$ (often in $L_2$ or $\ell$0), leading to: $\ell$1 The minimax risk thus scales as $\ell$2, where $\ell$3 solves the balance equation between sample size and local function class complexity (Shrotriya et al., 2022). This principle applies broadly to regression, density estimation, nonparametric location-scale models, and convex density classes.

Examples of entropy-driven rates:

Class	Covering Entropy $\ell$4	Minimax Rate $\ell$5
Hölder-$\ell$6 densities	$\ell$7	$\ell$8
TV-bounded densities	$\ell$9	$$R_n^* = \inf_{\hat{\theta}} \sup_{\theta \in \Theta} \mathbb E_\theta[\ell(\hat{\theta}, \theta)]$$0
Convex $$R_n^* = \inf_{\hat{\theta}} \sup_{\theta \in \Theta} \mathbb E_\theta[\ell(\hat{\theta}, \theta)]$$1-mixture simplex	$$R_n^* = \inf_{\hat{\theta}} \sup_{\theta \in \Theta} \mathbb E_\theta[\ell(\hat{\theta}, \theta)]$$2	$$R_n^* = \inf_{\hat{\theta}} \sup_{\theta \in \Theta} \mathbb E_\theta[\ell(\hat{\theta}, \theta)]$$3

3. Classical Minimax Rates in Nonparametric and High-Dimensional Problems

For nonparametric regression on Hölder or Sobolev classes, the optimal rate for mean-squared error is: $$R_n^* = \inf_{\hat{\theta}} \sup_{\theta \in \Theta} \mathbb E_\theta[\ell(\hat{\theta}, \theta)]$$4 where $$R_n^* = \inf_{\hat{\theta}} \sup_{\theta \in \Theta} \mathbb E_\theta[\ell(\hat{\theta}, \theta)]$$5 is the smoothness parameter and $$R_n^* = \inf_{\hat{\theta}} \sup_{\theta \in \Theta} \mathbb E_\theta[\ell(\hat{\theta}, \theta)]$$6 is dimensionality (O'Reilly et al., 2021, Mourtada et al., 2018, Zhao et al., 2023). In sup-norm, a logarithmic factor appears: $$R_n^* = \inf_{\hat{\theta}} \sup_{\theta \in \Theta} \mathbb E_\theta[\ell(\hat{\theta}, \theta)]$$7 In inverse problems, rates reflect both smoothness and operator ill-posedness; for Sobolev-type ellipsoid source sets with singular value decay $$R_n^* = \inf_{\hat{\theta}} \sup_{\theta \in \Theta} \mathbb E_\theta[\ell(\hat{\theta}, \theta)]$$8,

$$R_n^* = \inf_{\hat{\theta}} \sup_{\theta \in \Theta} \mathbb E_\theta[\ell(\hat{\theta}, \theta)]$$9

(Ding et al., 2017). In cost-sensitive and margin-sensitive classification on manifolds,

$L_2$0

where $L_2$1 is the margin exponent, and $L_2$2 is the intrinsic dimension (Reeve et al., 2018).

For sparse high-dimensional models (e.g., estimation in the Gaussian sequence model),

$L_2$3

with a phase transition ("elbow") at $L_2$4 separating sparse and dense regimes (Collier et al., 2015).

4. Structural Bias–Variance Decomposition and Geometry

Optimal estimation procedures balance geometric bias and variance, as formalized via random geometric partition statistics (e.g., diameters of tessellation cells, number of partition elements) (O'Reilly et al., 2021, Mourtada et al., 2018). For piecewise-constant estimators (histograms, forests), bias scales with average cell diameter (controlled by partition complexity), while variance is driven by sample allocation among cells.

In random forests built via stochastic tessellations (STIT, Poisson–hyperplane, Mondrian), optimal rates arise from balancing bias $L_2$5 and variance $L_2$6, yielding the tradeoff $L_2$7 (O'Reilly et al., 2021). Self-consistency and stationarity ensure that tessellation statistics scale appropriately under geometric homogeneity.

5. Minimax Rates in Random Forests: Axis-Aligned vs. Oblique Splits

Originally, minimax rates for forests were established only for axis-aligned Mondrian forests. Recent advances (O'Reilly et al., 2021) prove that fully oblique random tessellation forests (with arbitrary directional distributions $L_2$8) achieve identical minimax rates in arbitrary dimension due to the invariance of typical cell geometry and critical bias‐variance balancing. Specifically,

• For $L_2$9 (Hölder smoothness), $L_1$0.
• For $L_1$1 (one extra derivative), rate improves to $L_1$2 given sufficient averaging. These results demonstrate that oblique splits, favored empirically, retain the full minimax optimality of axis-aligned variants.

6. Extensions: Robustness, Adversarial Regimes, and Functional Estimation

Minimax theory extends to robust estimation under adversarial perturbations. For nonparametric regression subject to adversarial input attacks, the minimax rate is the sum of the standard estimation rate and the adversarial function deviation under the perturbation set: $L_1$3 with procedures such as the adversarial plug-in attaining this bound (Peng et al., 2024).

In regression under heavy-tailed, heteroskedastic, or non-Gaussian errors, minimax rates are determined by packing entropy of the regression function class, independent of the error law (subject to mild Hellinger differentiability conditions) (Zhao et al., 2023).

For functional estimation, minimax rates may display elbows or interpolation phenomena. In heterogeneous causal effect estimation, the optimal rate is dictated by the combined smoothness of nuisance and target functions, leading to a split between regression-like and functional-like rates: $L_1$4 for sufficient smoothness, with a slower rate $L_1$5 otherwise (Kennedy et al., 2022).

7. Network Analysis, Community Detection, and Testing

In network estimation problems (community detection, graphon estimation), minimax rates may be exponential rather than polynomial. In the Stochastic Block Model, the minimax misclassification error decays as: $L_1$6 highlighting a threshold phenomenon for strong vs. weak consistency (Zhang et al., 2015, Gao et al., 2018). Robust recovery under adversarial node corruptions preserves these rates up to additive error terms (Liu et al., 2022).

Similarly, for high-dimensional changepoint detection, minimax testing rates exhibit phase transitions between sparse and dense regimes, with explicit dependence on dimensionality, sparsity, and sample size, and unusual triple-logarithmic factors in certain regimes (Liu et al., 2019).

8. Time-Robust Minimax Rates and Sample Size Adaptivity

Classical minimax rates assume a fixed sample size. Time-robust minimax rates generalize to settings with uncertain or data-dependent sample size (anytime-valid estimation). In most problems, the time-robust rate differs from the classical rate by at most a logarithmic (or iterated-logarithmic) factor, e.g.,

$L_1$7

or for regular exponential families, $L_1$8 (Kirichenko et al., 2020). In model selection, time-robust rates enable simultaneous consistency and rate optimality, circumventing classical tradeoffs (AIC–BIC dilemma).

9. Practical Algorithmic Attainment and Adaptive Procedures

Rate-optimal estimators are often constructed by balancing geometric or combinatorial complexities:

• Sieve MLEs and multistage aggregation schemes adaptively achieve minimax rates across a range of function classes (Shrotriya et al., 2022).
• Random forests (Mondrian, STIT, Poisson–hyperplane) attain minimax rates via proper tuning of partition complexity and ensemble size (O'Reilly et al., 2021, Mourtada et al., 2018).
• Adaptive procedures (e.g., model aggregation, convex hulls) select near-oracle complexity in practice without prior smoothness knowledge.

10. Summary Table: Prototypical Minimax Rates

Setting	Rate $L_1$9	Reference
Hölder regression	$R_n^*$0	(O'Reilly et al., 2021)
Location-scale regression	$R_n^*$1	(Zhao et al., 2023)
Convex density class	$R_n^*$2~[Le Cam Eqn]	(Shrotriya et al., 2022)
Sparse vector estimation	$R_n^*$3	(Collier et al., 2015)
Graphon estimation	$R_n^*$4	(Gao et al., 2018)
SBM community detection	$R_n^*$5	(Zhang et al., 2015)
Adversarial regression	std. rate $R_n^*$6 max deviation	(Peng et al., 2024)
Inverse problems	$R_n^*$7	(Ding et al., 2017)
Causal effect estimation	$R_n^*$8, $R_n^*$9 elbow, see text	(Kennedy et al., 2022)

• "Minimax Rates for High-Dimensional Random Tessellation Forests" (O'Reilly et al., 2021)
• "Minimax rates of convergence for nonparametric location-scale models" (Zhao et al., 2023)
• "Minimax rates for robust community detection" (Liu et al., 2022)
• "Minimax rates for finite mixture estimation" (Heinrich et al., 2015)
• "Minimax rates for homology inference" (Balakrishnan et al., 2011)
• "Minimax rates in sparse, high-dimensional changepoint detection" (Liu et al., 2019)
• "Minimax rates of convergence for nonparametric regression under adversarial attacks" (Peng et al., 2024)
• "The minimax learning rates of normal and Ising undirected graphical models" (Devroye et al., 2018)
• "^^^^10^^^^" (Bilodeau et al., 2021)
• "Minimax rates without the fixed sample size assumption" (Kirichenko et al., 2020)
• "Minimax rates in network analysis: graphon estimation, community detection and hypothesis testing" (Gao et al., 2018)
• "Minimax manifold estimation" (Genovese et al., 2010)
• "Minimax optimal rates for Mondrian trees and forests" (Mourtada et al., 2018)
• "Minimax rates for statistical inverse problems under general source conditions" (Ding et al., 2017)
• "Minimax rates of community detection in stochastic block models" (Zhang et al., 2015)
• "Minimax rates for cost-sensitive learning on manifolds with approximate nearest neighbours" (Reeve et al., 2018)
• "Revisiting Le Cam’s Equation: Exact Minimax Rates over Convex Density Classes" (Shrotriya et al., 2022)
• "Minimax learning rates for estimating binary classifiers under margin conditions" (García et al., 15 May 2025)
• "Minimax rates for heterogeneous causal effect estimation" (Kennedy et al., 2022)