Diagonal Plus Low-Rank Parameterization

• DPLR parameterization is a matrix modeling approach that decomposes a matrix into a diagonal (local variability) and a low-rank (global structure) component, improving interpretability and performance.
• It finds applications in covariance estimation, precision matrices, operator approximations, and recommendation systems, offering scalable and efficient computational methods.
• Advanced estimation techniques, including blockwise coordinate descent and convex optimization, ensure precise recovery with robust error bounds in high-dimensional settings.

The Diagonal-Plus-Low-Rank (DPLR) parameterization refers to the modeling, analysis, and estimation of matrices—typically covariance, precision, kernel, or operator matrices—as a sum of a diagonal matrix and a low-rank matrix. The core structural assumption is that the target matrix exhibits both (a) global shared structure (captured by the low-rank term) and (b) localized or per-coordinate variability (captured by the diagonal). This structure is ubiquitous in high-dimensional statistics, kernel learning, signal processing, matrix approximation, machine learning systems, and large-scale scientific computing.

1. Mathematical Formulation and Variants

DPLR decomposes a symmetric or Hermitian matrix $M \in \mathbb{R}^{n \times n}$ as

$M = D + L$

where $D$ is a diagonal matrix ($D = \mathrm{diag}(d_1, \ldots, d_n)$), and $L$ is a low-rank positive semidefinite (PSD) matrix ($\mathrm{rank}(L) \le r \ll n$). For general linear operators or kernel matrices, $L$ may be factored as $UV^T$ or $UU^T$.

Several commonly encountered DPLR forms include:

• Covariance estimation: $\Sigma_0 = L_\Sigma + D_\Sigma$, where $\Sigma_0$ is a population covariance, $L_\Sigma$ PSD and low-rank, $D_\Sigma$ diagonal and positive-definite (Wu et al., 2018).
• Precision matrix (inverse covariance): $\Theta_0 = -L_0 + D_0$, with $L_0$ PSD, $D_0$ diagonal and positive-definite (Wu et al., 2018).
• Operator approximation: $M = D + UV^T$, with $U,V \in \mathbb{R}^{n \times r}$ (Fernandez et al., 28 Sep 2025).
• Factor analysis: $\Sigma = D + LL^T$, $D \succeq 0$ diagonal, $L \in \mathbb{R}^{p \times r}$ (Khamaru et al., 2018).

This parameterization generalizes both pure low-rank (global factor) and diagonal (coordinate-wise structure) models, and is strictly more expressive than either in isolation.

2. Identifiability and Convex Recovery

Identifiability within DPLR models depends crucially on the column-space of the low-rank term. Classical criteria for uniqueness of DPLR decomposition include:

• Balance condition: In one dimension, a vector $u$ is "balanced" if $|u_i| \le \sum_{j\neq i} |u_j|$ for all $i$; this is necessary and sufficient for recoverability (Saunderson et al., 2012).
• Coherence threshold: For a subspace $\mathcal{U}$, the coherence $$\mu(\mathcal{U}) = \max_i \| P_{\mathcal{U}} e_i \|^2$$; $\mathcal{U}$ is identifiable if $\mu(\mathcal{U}) < \frac12$ (Saunderson et al., 2012).

Convex optimization procedures such as Minimum Trace Factor Analysis (MTFA), which replaces direct rank minimization with trace minimization, recover DPLR decompositions under suitable identifiability and coherence conditions: $$\min_{D,L} \mathrm{tr}(L) \quad \text{ s.t. } \quad X = D + L, \; L \succeq 0, \; D \text{ diagonal}$$ For moderate $n$, SDPT3, SeDuMi, and MOSEK suffice; for larger $n$, operator splitting and sketching methods exploit the structure (Saunderson et al., 2012).

The dual of this program is equivalent to searching for faces of the elliptope (the set of correlation matrices), directly linking DPLR recoverability, facial structure of $E_n = \{Y \succeq 0 : \mathrm{diag}(Y) = 1\}$, and geometric ellipsoid fitting (Saunderson et al., 2012).

3. Estimation Algorithms and Computational Procedures

DPLR estimation is implemented in a range of algorithms, including blockwise coordinate descent, spectral alternation, difference-of-convex optimization, and convex sketching.

• Blockwise Coordinate Descent: Alternates closed-form $L$-update (eigendecomposition) and $D$-update (log-det SDP or coordinate optimization). For covariance estimation,
  • $L$-step: $L = D^{1/2} U_{r} V U_{r}^T D^{1/2}$ with $U_{r},V$ from partial eigendecomposition.
  • $D$-step: minimizes $\mathrm{tr}((D-L)S) - \log|D-L|$ over diagonals (Wu et al., 2018).
• Alternating Spectral Decomposition: Alternates between low-rank fit (Eckart–Young best approximation to $\Sigma-D$) and diagonal update $D=\mathrm{diag}(\Sigma - UU^T)$. Randomized variants (Nyström for low-rank, Diag++ for diagonal) achieve scalability in the matrix-vector-product setting with rigorous error bounds (Yeon et al., 18 Dec 2025).
• Difference-of-Convex Factor Analysis (DC-FA): Rewrites the factor analysis ML estimator in terms of $(f_1 - f_2)$, where $f_1$ is coordinatewise convex and $f_2$ is a nonsmooth convex spectral function. Iterative majorization-minimization yields closed-form update rules per coordinate exploiting spectral subgradients (Khamaru et al., 2018).
• Convex Sketched Estimation: In sketching-based recovery, SKETCHLORD solves

$$\min_{D,U,V} \frac12 \| A - (D + UV^T) \|_F^2 + \lambda \|UV^T\|_*$$

via matrix sketches and nuclear norm regularization, attaining exact low-rank plus diagonal recovery in high-dimensional operator settings (Fernandez et al., 28 Sep 2025).

4. Theoretical Guarantees and Error Analysis

Theoretical properties of DPLR estimation encompass consistency, monotonicity, error bounds, and computational complexity:

• Consistency of Estimators: For high-dimensional covariance estimation, when $\Sigma_0$ truly decomposes as $L+D$ with $\mathrm{rank}(L)\le r_0$, both fixed-rank and rank-penalized estimators satisfy

$$\| \widehat{\Theta} - \Theta_0 \|_F = O_p\left( \max\{ \sqrt{r_0 p/n},\, \sqrt{(p \log p)/n} \} \right)$$

matching rates of sparse-precision estimators (Wu et al., 2018).

• Monotone Frobenius Decrease: Alternating algorithms decrease the approximation error monotonically, with local contraction in neighborhoods of true parameters (Yeon et al., 18 Dec 2025).
• Randomized Sketching Guarantees: Sketchlord achieves exact recovery for idealized DPLR matrices and outperforms any sequential diagonal-then-low-rank or low-rank-then-diagonal pipeline, provided sketch dimension is $p \gg r$ (Fernandez et al., 28 Sep 2025). Non-asymptotic error bounds from matrix concentration theory apply.

5. Practical Implementations and Applications

DPLR parameterizations arise naturally in diverse application domains:

• Covariance and Precision Estimation: DPLR estimators outperform graphical lasso in KL-divergence loss, especially when the true structure is low-rank plus diagonal. In financial portfolios (Markowitz allocation), DPLR-based covariance yields portfolios with lower volatility and higher Sharpe ratio than sparse or diagonal-only benchmarks (Wu et al., 2018).
• Factorization Machines for Recommendation: DPLR-decomposed field-weight matrices in FwFM architectures reduce inference and training costs from $O(m^2k)$ to $O(mrk)$, outperforming aggressive pruning in both accuracy and latency on large-scale ad ranking (Shtoff et al., 2024).
• Eigenproblems and Polynomial Rootfinding: DPLR Hessenberg reduction via quasiseparable generators delivers $O(n^2 k)$ complexity solvers for secular companion linearizations, with norm-wise backward-stability and machine-precision eigenvalue accuracy (Bini et al., 2015).
• Large-Scale PDEs and Filtering: DPLR Riccati projection enables tractable ODE integration of very large PSD flows, preserving invertibility and enabling linear-time inversion via Woodbury identity—crucial for high-dimensional Kalman filters and Bayesian inference (Bonnabel et al., 2024).
• High-Dimensional Operator Sketching: Sketchlord enables fast, high-fidelity recovery of deep learning Hessians as DPLR objects from $O(r)$ matrix-vector products, matching empirical spectra and outperforming sequential approaches (Fernandez et al., 28 Sep 2025).

6. Methodological Considerations and Extensions

Optimization and estimation strategies for DPLR models reflect tradeoffs in computational complexity, rank selection, and regularization:

• Rank selection: Penalized likelihood, cross-validation, and ablations on operator residuals are used to select rank $r$.
• Regularization: $\ell_2$ or trace-norm penalties on both diagonals and low-rank factors stabilize training and estimation.
• Structure extension: Methodologies generalize to block-diagonal plus low-rank, banded plus low-rank, or block-sparse plus factor models.
• Dynamic adaptation: In ODE flows, one may adjust rank or diagonal structure dynamically based on error monitoring (Bonnabel et al., 2024).
• Hyperparameter tuning: Sketch dimension, regularization weights, and learning rates are tuned by application-specific canary tests and cross-validation (Shtoff et al., 2024, Fernandez et al., 28 Sep 2025).

7. Comparative Analysis and Empirical Performance

Empirical studies consistently demonstrate that DPLR estimators, when the true matrix exhibits both global and local structure, outperform pure low-rank, pure diagonal, and sequential hybrid approximations. Key performance attributes include:

• Accuracy: Lower approximation error and improved metrics (AUC, LogLoss, KL-loss) relative to comparator methods.
• Scalability: Linear or near-linear scaling in ambient dimension $n$ or number of fields $m$, crucial for both scientific and commercial datasets.
• Efficiency: Reduced memory and compute footprint, especially when $r \ll n$, enabling deployment in latency-constrained data paths or massive-scale inference engines (Shtoff et al., 2024, Fernandez et al., 28 Sep 2025).
• Numerical stability: DPLR methods maintain backward-stability and controlled error propagation in large-scale linear algebra (Bini et al., 2015).

The DPLR parameterization is thus foundational in modern high-dimensional statistics, scalable operator approximation, kernel learning, and recommendation systems, with well-understood identifiability, efficient computational frameworks, and empirical superiority in complex real-world applications.