Well-Conditioned Oblivious Perturbations in Linear Space

Abstract: Perturbing a deterministic $n$-dimensional matrix with small Gaussian noise is a cornerstone of smoothed analysis of algorithms [Spielman and Teng, JACM 2004], as it reduces the condition number of the input to $O(n)$, and with it the complexity of many matrix algorithms. However, when deployed algorithmically, these perturbations are expensive due to the cost of generating and storing $n^2$ Gaussian random variables. We propose a perturbation that requires generating and storing $O(n)$ random numbers in $O(\log n)$ bits of precision, and reduces the condition number of any deterministic matrix to $O(n)$, matching Gaussian perturbations. Our result in particular implies a better complexity for the perturbed conjugate gradient algorithm, showing that we can solve an $n\times n$ linear system in linear space to within an arbitrarily small constant backward error using $O(n)$ matrix-vector products. In our construction, we introduce the concept of a pattern matrix, which is a dense deterministic matrix that maps all sparse vectors into dense vectors, and we combine it with a sparse perturbation whose entries are dependent and located in a non-uniform fashion. In order to analyze this construction, we develop new techniques for lower bounding the smallest singular value of a random matrix with dependent entries.

• The paper introduces an oblivious perturbation method that generates well-conditioned matrices using O(n log n) random bits and linear space.
• It combines a deterministic pattern matrix with a sparse random perturbation to overcome the coupon-collector barrier and ensure optimal conditioning.
• The approach enables backward-stable, space-efficient solvers for linear systems with improved matrix-vector query complexity.

Well-Conditioned Oblivious Perturbations Using Near-Optimal Randomness and Space

Problem Context and Motivation

Smoothed analysis has emerged as a core analytic framework for bridging worst-case and typical-case performance of algorithms, particularly in computational linear algebra. The canonical procedure of perturbing an arbitrary matrix with small Gaussian noise induces well-conditioned inputs with condition number $O(n)$ and underpins both theoretical guarantees and empirical algorithmic stability [Spielman and Teng, JACM 2004]. However, dense Gaussian perturbations entail $O(n^2)$ random variables and comparable space complexity, which can dwarf the requirements of structured or sparse data and render such approaches impractical for large-scale or memory-constrained applications.

This work addresses the central open question: Can one construct a random matrix perturbation that, with high probability, well-conditions any deterministic input matrix, but requires only $O(n)$ space and near-linear (in $n$) random bits, matching (up to logarithmic factors) the input size?

Main Results

The authors introduce a new class of oblivious perturbations: distributions over $n \times n$ random matrices $R$ such that, for any unit-norm deterministic matrix $A$, with high probability, $A + \varepsilon R$ is well-conditioned (condition number $O(n)$) and $R$ itself has operator norm at most 1. The construction satisfies:

• Random Bit Complexity: The perturbation is generated from only $O(n^2)$0 random bits, nearly matching the information-theoretic lower bound for non-singularity.
• Linear Space: The perturbation can be represented in $O(n^2)$1 numbers of $O(n^2)$2 bits each.
• Obliviousness: The random matrix $O(n^2)$3 is sampled independently of $O(n^2)$4 and works for arbitrary deterministic input.
• Optimal Conditioning: The resulting condition number is $O(n^2)$5, mirroring the outcome of dense Gaussian perturbations.

Theorem (informal): Using $O(n^2)$6 random bits, one can generate an $O(n^2)$7 oblivious, well-conditioned perturbation $O(n^2)$8 such that for all $O(n^2)$9 with $O(n)$0, with probability $O(n)$1, $O(n)$2.

Construction and Technical Approach

The construction of $O(n)$3 is a sum of two independent random matrices: a “pattern matrix” based component and a “sparse” random perturbation. The approach overcomes several key obstacles in prior work, specifically the coupon-collector barrier for sparsity and the difficulty of lower bounding the smallest singular value in the presence of dependent entries.

Pattern Matrix

• Role: Handles "compressible" (approximately sparse) vectors, which are the main source of instability for sparse perturbations.
• Structure: A deterministic $O(n)$4 matrix $O(n)$5 with two random diagonal sign flips—constructed using only $O(n)$6 random bits—that ensures for any sparse vector $O(n)$7, $O(n)$8 and $O(n)$9 are dense: a constant fraction of coordinates are bounded from zero.
• Key Property: For all sparse $n$0 and all sign patterns, a large number of coordinates of $n$1 exceed some fixed threshold.

Sparse Dependent Random Matrix

• Role: Controls the invertibility over "incompressible" (non-sparse) directions by directly randomizing a small set of entries.
• Construction: Each column picks $n$2 positions uniformly without replacement, and only these receive random signs. To maintain bounded operator norm, all rows with more than $n$3 nonzeros are zeroed out (“row trimming”).
• Technical Innovation: The row-trimming operation introduces dependence between entries. The analysis uses a sophisticated conditioning argument to establish the presence of enough spread coordinates in the final sparse matrix, allowing a nontrivial Littlewood-Offord-type lower bound for the smallest singular value.

Singular Value Analysis

The standard approach of decomposing the unit sphere into compressible and incompressible vectors is applied:

• Compressible vectors: The pattern matrix ensures that for all such vectors, the image under the total perturbation is not small, by a tensorization argument, leveraging independence of signs and the pattern matrix property.
• Incompressible vectors: The invertibility-by-distance method, adapted from random matrix theory, is refined to handle the dependencies induced by row trimming. The concentration of mass and the random placement of nonzeros enable the authors to prove, via advanced probabilistic arguments, that the smallest singular value is large with high probability.

Linear Space Backward-Stable Solvers

A direct consequence is an improved randomized algorithm for solving linear systems. Using matrix-vector access to $n$4 and $n$5, one can, with $n$6 random bits and $n$7 machine-word space, solve $n$8 to within arbitrarily small backward error after $n$9 (rather than $n \times n$0) matrix-vector queries, even in finite-precision arithmetic:

• Perturbed Conjugate Gradient: By pre-adding the described perturbation, the condition number—as seen by CG—is $n \times n$1, so $n \times n$2 steps suffice to reach the backward error threshold, using only linear storage and random bits.
• Theoretical Scope: This is the first method that, in finite-precision and with linear space, achieves $n \times n$3 matrix-vector product complexity with high-probability backward error guarantees for arbitrary square linear systems.

Comparison with Prior Work

Prior proposals for sparse or low-randomness perturbations either required superlinear random bits, yielded suboptimal (superpolynomial in $n \times n$4) condition numbers due to coupon-collector phenomena, or could only address limited structural settings (e.g., diagonal shifts or low-rank perturbations).

• The method of Shah, Srivastava, and Zeng requires $n \times n$5 nonzeros and cannot achieve both $n \times n$6 condition number and $n \times n$7 random bits simultaneously.
• This work is the first to break the sparsity bottleneck, combining ideas from derandomized embeddings [Lovett and Sodin] and dependent random design, and provides an explicit constructive answer matching the Gaussian regime.

Implications and Future Directions

This work offers both a theoretical advance in derandomizing smoothed analysis and a directly implementable algorithm for regularizing ill-conditioned systems in space-limited computation. The hybridization of deterministic dense structures and minimal, highly structured randomness should inspire further research in low-randomness complexity resources for smoothed and average-case analyses across numerical linear algebra and optimization.

Potential further developments include:

• Extension to Overdetermined/Underdetermined or Structured Systems: Adapting pattern matrix constructions to other matrix ensembles (e.g., rectangular, block, or Toeplitz).
• Explicit Derandomization: Moving toward fully deterministic constructions or even further reducing the random bits via advanced pseudorandomness techniques.
• Broader Applications: Employing such perturbations for derandomized spectral clustering, kernel approximation, and randomized stability in eigenvalue computations.

Conclusion

The paper introduces a rigorously analyzed and practical scheme for well-conditioned oblivious perturbations in linear space and near-minimal randomness, resolving a fundamental challenge in smoothed analysis and finite-precision numerical linear algebra. The analysis leverages new probabilistic and combinatorial arguments for dependent sparsification and structured randomness. The results enable backward-stable, space-efficient solvers for linear systems and open new avenues for low-randomness, high-stability computations in high dimension.

---

Citation:

"Well-Conditioned Oblivious Perturbations in Linear Space" (2604.23193)