Cross-Fitted Norm-Truncated Estimator

• The paper introduces the Cross-Fitted Norm-Truncated Estimator, a novel method achieving near-optimal covariance estimation for Sub-Weibull data with minimal computational cost.
• Its norm-truncation operator preserves spectral geometry and guarantees positive semidefiniteness, addressing challenges posed by heavy-tailed distributions.
• The cross-fitting strategy and tight non-asymptotic bounds ensure robustness and scalability, outperforming iterative M-estimation methods in high-dimensional settings.

The Cross-Fitted Norm-Truncated Estimator is a computationally efficient method for estimating covariance matrices in high-dimensional settings where the data exhibit Sub-Weibull tail behavior. Unlike classical robust estimators that rely on iterative M-estimation or semidefinite programming, this approach is specifically constructed to address the statistical and computational challenges posed by heavy-tailed, stretched-exponential distributions, achieving theoretical optimality with minimal computational cost (He, 19 Dec 2025).

1. Sub-Weibull Distributions and Covariance Estimation

Sub-Weibull random vectors $X \in \mathbb{R}^d$ are defined by stretched-exponential tail decay. Explicitly, $X$ is Sub-Weibull of order $\alpha > 0$ (denoted $X \in \mathrm{SubW}(\alpha, \lambda)$) if there exist constants $C, \lambda > 0$ such that the Euclidean norm obeys

$$P(\|X\|_2 > t) \leq C \exp\left[-(t/\lambda)^\alpha\right] \quad \forall t \geq 0.$$

For all $p \geq 1$ and unit vectors $u$, the moments satisfy

$$(E|u^\top X|^p)^{1/p} \leq C' \lambda p^{1/\alpha} \|u\|_2,$$

ensuring that all moments grow at most like $p^{1/\alpha}$.

Covariance estimation for Sub-Weibull data is challenging due to sensitivity to outliers and elevated probability of extreme observations relative to Gaussian settings. Traditional estimators often fail to maintain both statistical performance and computational tractability. The Cross-Fitted Norm-Truncated Estimator (CF-NTE) directly targets these limitations.

2. Norm-Truncation Operator and Mechanism

The core filtering mechanism is the norm-truncation operator:

$$\psi_\tau(x) = x \cdot 1_{\|x\|_2 \leq \tau} + \tau \frac{x}{\|x\|_2} \cdot 1_{\|x\|_2 > \tau}.$$

This function projects any vector $x \in \mathbb{R}^d$ onto the ball of radius $\tau$ in Euclidean (radial) fashion, retaining direction while capping maximum length at $\tau$.

Element-wise truncation (coordinate-wise winsorization) is subsumed by this approach; however, the norm-truncation operator preserves the spectral geometry and positive semidefiniteness (PSD), unlike coordinate-wise methods which typically disrupt joint geometric structure and can produce non-PSD covariances (He, 19 Dec 2025).

3. Cross-Fitting Scheme and Estimator Construction

To avoid circularity in selecting the truncation radius $\tau$, a cross-fitting strategy is implemented for $N$ samples $\{X_i\}$:

1. Randomly partition data into two (approximately) equal folds $S_1, S_2$.
2. For $S_1$, compute the sample median of norms $\{\|\;X_i\|_2 : i \in S_1\}$, denoted $\hat{\theta}_1$. Set $\tau_1 = C \hat{\theta}_1 (\log N)^{1/\alpha}$, where $C > 0$ is tuned so $P(\|X\|_2 > \tau_1) \lesssim N^{-4}$.
3. Apply $\psi_{\tau_1}$ to each sample in $S_2$ and compute the truncated empirical covariance:

$$\hat{\Sigma}^{(2)} = \frac{1}{|S_2|} \sum_{j \in S_2} \psi_{\tau_1}(X_j) \psi_{\tau_1}(X_j)^T.$$

1. Repeat steps 2–3, swapping $S_1$ and $S_2$.

The final aggregated estimator is

$$\hat{\Sigma}_{\mathrm{cf}} = \frac{1}{2} (\hat{\Sigma}^{(1)} + \hat{\Sigma}^{(2)}).$$

The construction guarantees independence between sample splits and truncation thresholds, enabling rigorous application of concentration inequalities.

In compact form:

$$\hat{\Sigma}_{\mathrm{cf}} = \frac{1}{N} \sum_{i=1}^N \psi_{\tau(i)}(X_i) \psi_{\tau(i)}(X_i)^T,$$

where $\tau(i) = \tau_1$ for $i \in S_2$ and $\tau(i) = \tau_2$ otherwise.

4. Algorithmic Complexity and Implementation

The algorithmic steps are:

• Compute Euclidean norms: $O(Nd)$.
• Median selection: $O(N)$.
• Covariance construction (two folds): $O(Nd^2)$.

Total complexity is dominated by $O(Nd^2)$, matching the theoretical lower bound for forming a $d \times d$ covariance matrix. No iterative refinement or matrix decompositions are required. This facilitates scalability when $d \gg 1000$, and iterative methods (M-estimation, semidefinite programming) with at least $O(d^3)$ cost are computationally prohibitive (He, 19 Dec 2025).

5. Statistical Guarantees and Non-Asymptotic Bounds

Under the assumptions:

• $(A1)$ Each coordinate of $Z \sim \Sigma^{-1/2} X$ satisfies $\|Z_j\|_{\psi_\alpha} \leq K$ (uniform Orlicz norm bound).
• $(A2)$ The effective rank $$r(\Sigma) = \mathrm{tr}(\Sigma)/\|\Sigma\|_{\mathrm{op}} \geq C_0 \log N$$.

With high probability $(\geq 1 - O(N^{-1}))$,

$$\|\hat{\Sigma}_{\mathrm{cf}} - \Sigma\|_{\mathrm{op}} \leq C_1 \|\Sigma\|_{\mathrm{op}} \left( \sqrt{\frac{r(\Sigma) \log N}{N}} + \frac{r(\Sigma) (\log N)^{1 + 2/\alpha}}{N} \right).$$

For sufficiently large $N$, the second term becomes negligible compared to the first, yielding the sub-Gaussian rate for operator norm error:

$$\|\hat{\Sigma}_{\mathrm{cf}} - \Sigma\|_{\mathrm{op}} = O\left( \|\Sigma\|_{\mathrm{op}} \sqrt{\frac{r(\Sigma) \log N}{N}} \right).$$

Proof leverages weighted Hanson-Wright inequalities for Sub-Weibull vectors and matrix-Bernstein concentration for truncated quadratic forms (He, 19 Dec 2025).

6. Comparative Evaluation with Alternative Robust Covariance Estimators

A comparison of robust covariance estimators under heavy-tailed regimes is summarized in the table below.

Estimator Type	PSD Preserved	Rate ($\|\cdot\|_{\mathrm{op}}$)	Computational Cost
Element-wise truncation	No	Suboptimal	$O(Nd^2)$
M-Estimators	Yes	Optimal ($O(\sqrt{r(\Sigma)/N})$)	$O(TNd^2)$ or $O(d^3)$ (iter.)
Spectrum-wise truncation (SVD)	Yes	Unreliable under principal drift	$O(d^3)$
Cross-Fitted Norm-Truncation	Yes	Optimal	$O(Nd^2)$

Element-wise truncation destroys joint geometry and does not guarantee positive semidefiniteness. M-estimators (e.g., Catoni’s, Geometric Median) achieve statistically optimal rates but are computationally intensive. Spectrum-wise truncation requires an SVD and can fail under significant outlier-induced principal rotation. The Cross-Fitted Norm-Truncated Estimator neither requires iterative optimization nor SVD and maintains the essential geometric structure.

7. Practical Implementation and Use Cases

Selecting the truncation constant $C$ affects bias–variance performance; moderate values in $[2, 5]$ generally offer stability. The estimator exhibits low sensitivity to $C$ due to the flattening of bias–variance trade-off (“bathtub” shape), and improper choices can lead to either excess bias (too small $C$) or variance inflation (too large $C$).

The estimator is particularly well-suited for high-dimensional datasets with heavy tails—such as financial returns, network traffic, or large-scale biological measurements—where the tails decay faster than any polynomial but slower than Gaussian. It is ideal when robust, PSD covariances are required for downstream procedures (PCA, LDA, Gaussian graphical models) but computational constraints preclude iterative robust methods.

In summary, the Cross-Fitted Norm-Truncated Estimator provides near-optimal statistical robustness for Sub-Weibull vectors, preserves spectral structure, and achieves minimal computational complexity. It bridges the gap between tractable computation and outlier-resistant inference in modern high-dimensional data analysis (He, 19 Dec 2025).