Compact Householder-type Bidiagonal Updates

• Compact Householder-type bidiagonal updates are efficient algorithms that update matrix bidiagonal factorizations under low-rank modifications while preserving sparsity.
• They employ Householder reflectors and the compact WY representation to restore bidiagonality and maintain numerical stability in streaming data applications.
• This method significantly reduces memory usage and computational cost compared to full SVD recomputation, making it ideal for high-throughput scenarios.

A compact Householder-type bidiagonal update is an algorithmic technique for updating the bidiagonal factorization of a matrix under low-rank modifications, with particular emphasis on streaming data and high-throughput matrix applications. Such updates provide a memory-efficient and numerically stable alternative to full singular value decomposition (SVD) recomputation, decoupling a sparse component from the low-rank update and enabling efficient maintenance of bidiagonal structure even under frequent modifications (Brust et al., 2 Sep 2025).

1. Mathematical Formulation

Let $A \in \mathbb{R}^{m\times n}$ be a matrix at step $k$ of a data stream with a known bidiagonal factorization:

$A = Q B P$

where $Q \in \mathbb{R}^{m \times m}$ and $P \in \mathbb{R}^{n \times n}$ are orthogonal, and $$B = \operatorname{bidiag}(\alpha_1,\ldots,\alpha_t; \beta_1,\ldots,\beta_{t-1}) \in \mathbb{R}^{m \times n}$$ is upper bidiagonal, $t=\min(m,n)$.

Given a low-rank update $Â = A + B C$ with $B \in \mathbb{R}^{m \times r}$, $C \in \mathbb{R}^{n \times r}$ (rank-$k$0), one transforms the update into the framework of the existing factors:

$k$1

with $k$2 and $k$3.

In the rank-1 case ($k$4), $k$5 and the inner update to be bidiagonalized is $k$6, with $k$7, $k$8.

To restore bidiagonality, Householder reflectors $k$9 (left) and $A = Q B P$0 (right) are constructed to zero out subdiagonal and superdiagonal elements as required. Each reflector takes the form $A = Q B P$1 with scalar $A = Q B P$2 and appropriate support. The compact WY representation accumulates these reflectors: for the left, $A = Q B P$3, $A = Q B P$4 with entries $A = Q B P$5 for $A = Q B P$6, $A = Q B P$7 for $A = Q B P$8 and $A = Q B P$9 otherwise; $Q \in \mathbb{R}^{m \times m}$0.

Similarly, right reflectors $Q \in \mathbb{R}^{m \times m}$1, $Q \in \mathbb{R}^{m \times m}$2: $Q \in \mathbb{R}^{m \times m}$3.

Applying $Q \in \mathbb{R}^{m \times m}$4 pairs of reflectors to $Q \in \mathbb{R}^{m \times m}$5 yields the key factorization:

$Q \in \mathbb{R}^{m \times m}$6

with $Q \in \mathbb{R}^{m \times m}$7 structured so that the corrections are rank-$Q \in \mathbb{R}^{m \times m}$8. Because $Q \in \mathbb{R}^{m \times m}$9 is bidiagonal and the update is low-rank, fill-in is limited, maintaining sparsity of the factors.

2. Algorithmic Procedure

The compact Householder-type bidiagonal update algorithm (denoted in (Brust et al., 2 Sep 2025) as BHU) proceeds as follows for the rank-1 case:

1. Initialization: Start with $P \in \mathbb{R}^{n \times n}$0, $P \in \mathbb{R}^{n \times n}$1, $P \in \mathbb{R}^{n \times n}$2, and $P \in \mathbb{R}^{n \times n}$3, $P \in \mathbb{R}^{n \times n}$4 implicit via $P \in \mathbb{R}^{n \times n}$5, $P \in \mathbb{R}^{n \times n}$6.
2. Transform Update Vectors: Set $P \in \mathbb{R}^{n \times n}$7 (initially just $P \in \mathbb{R}^{n \times n}$8), $P \in \mathbb{R}^{n \times n}$9.
3. Iterative Bidiagonalization: For $$B = \operatorname{bidiag}(\alpha_1,\ldots,\alpha_t; \beta_1,\ldots,\beta_{t-1}) \in \mathbb{R}^{m \times n}$$0 to $$B = \operatorname{bidiag}(\alpha_1,\ldots,\alpha_t; \beta_1,\ldots,\beta_{t-1}) \in \mathbb{R}^{m \times n}$$1:
   • Compute the $$B = \operatorname{bidiag}(\alpha_1,\ldots,\alpha_t; \beta_1,\ldots,\beta_{t-1}) \in \mathbb{R}^{m \times n}$$2th transformed column $$B = \operatorname{bidiag}(\alpha_1,\ldots,\alpha_t; \beta_1,\ldots,\beta_{t-1}) \in \mathbb{R}^{m \times n}$$3.
   • Form Householder $$B = \operatorname{bidiag}(\alpha_1,\ldots,\alpha_t; \beta_1,\ldots,\beta_{t-1}) \in \mathbb{R}^{m \times n}$$4 to annihilate $$B = \operatorname{bidiag}(\alpha_1,\ldots,\alpha_t; \beta_1,\ldots,\beta_{t-1}) \in \mathbb{R}^{m \times n}$$5, append to $$B = \operatorname{bidiag}(\alpha_1,\ldots,\alpha_t; \beta_1,\ldots,\beta_{t-1}) \in \mathbb{R}^{m \times n}$$6.
   • Update left-block $$B = \operatorname{bidiag}(\alpha_1,\ldots,\alpha_t; \beta_1,\ldots,\beta_{t-1}) \in \mathbb{R}^{m \times n}$$7.
   • Compute transformed row and corresponding Householder $$B = \operatorname{bidiag}(\alpha_1,\ldots,\alpha_t; \beta_1,\ldots,\beta_{t-1}) \in \mathbb{R}^{m \times n}$$8, append to $$B = \operatorname{bidiag}(\alpha_1,\ldots,\alpha_t; \beta_1,\ldots,\beta_{t-1}) \in \mathbb{R}^{m \times n}$$9.
   • Grow $t=\min(m,n)$0 per the structured formula.
4. Extraction: After $t=\min(m,n)$1, the updated bidiagonal $t=\min(m,n)$2 is the upper bidiagonal part of $t=\min(m,n)$3. Updated $t=\min(m,n)$4 and $t=\min(m,n)$5 can be stored implicitly via the compact reflectors.

This procedure circumvents dense intermediate matrices and internalizes the update in the Householder compact form. Each step updates the implicit $t=\min(m,n)$6 and $t=\min(m,n)$7 factors without materializing giant orthogonal arrays.

3. Complexity and Memory Analysis

Let $t=\min(m,n)$8. Each iteration involves:

• Forming the active column: $t=\min(m,n)$9 flops.
• Triangular solves with $Â = A + B C$0: $Â = A + B C$1.
• Householder operations: $Â = A + B C$2.

Summing over $Â = A + B C$3 to $Â = A + B C$4 yields a global computational cost of

$Â = A + B C$5

This complexity is about $Â = A + B C$6 higher than standard dense Householder bidiagonalization (as in LAPACK’s dgebrd), which costs $Â = A + B C$7.

Memory usage: $Â = A + B C$8, $Â = A + B C$9, vectors $B \in \mathbb{R}^{m \times r}$0, $B \in \mathbb{R}^{m \times r}$1, and triangular factors $B \in \mathbb{R}^{m \times r}$2, $B \in \mathbb{R}^{m \times r}$3 packed into $B \in \mathbb{R}^{m \times r}$4, $B \in \mathbb{R}^{m \times r}$5. The total storage is approximately $B \in \mathbb{R}^{m \times r}$6 words. This is about half of the extra storage required by blocked bidiagonalization and significantly less than that required by forming dense intermediate matrices in standard SVD updating approaches, such as Brand (2006).

4. Numerical Stability and Accuracy

Each Householder reflector is normwise backward-stable. The compact WY representation preserves orthogonality to working precision:

$B \in \mathbb{R}^{m \times r}$7

where $B \in \mathbb{R}^{m \times r}$8 is machine epsilon. The error in the reconstructed matrix after update satisfies:

$B \in \mathbb{R}^{m \times r}$9

The truncation error is governed by the Eckart–Young theorem adapted for bidiagonal matrices:

$C \in \mathbb{R}^{n \times r}$0

where $C \in \mathbb{R}^{n \times r}$1, $C \in \mathbb{R}^{n \times r}$2 are the diagonal and subdiagonal elements of the updated $C \in \mathbb{R}^{n \times r}$3.

Occasional re-orthogonalization of $C \in \mathbb{R}^{n \times r}$4, $C \in \mathbb{R}^{n \times r}$5 may be warranted for accumulated backward error. The cost of re-orthogonalization is $C \in \mathbb{R}^{n \times r}$6.

5. Illustrative Example

For $C \in \mathbb{R}^{n \times r}$7, let $C \in \mathbb{R}^{n \times r}$8 be initially

$C \in \mathbb{R}^{n \times r}$9

and vectors $k$00, $k$01. Stepwise:

1. First Householder ($k$02):

$k$03

Choose $k$04 to zero $k$05: $k$06, $k$07.

2. First right Householder: Compute row $k$08; eliminate further entries via $k$09.

Continuing for $k$10 yields the full bidiagonalization with updated $k$11, $k$12 in $k$13. For the concrete choice $k$14, $k$15, $k$16, the update $k$17 increases $k$18 only: $k$19 is unchanged except $k$20.

6. Practical Context and Comparative Perspective

The compact Householder-type bidiagonal update as outlined in (Brust et al., 2 Sep 2025) is specifically effective for high-throughput scenarios such as recommendation systems and network analysis where rapid, low-memory, and accurate subspace tracking is critical. The approach is contrasted to LAPACK’s dense algorithms and incremental SVD updates, offering qualitatively similar accuracy with reduced intermediate memory requirements. A primary distinction is that the compact Householder approach trades some additional floating-point operations for dramatically reduced workspace, which is often the limiting resource in large-scale data settings.

Theoretical and empirical analysis establishes parity to SVD-based accuracy in practice, with the added benefit of explicit control of orthogonality, stable backward error, and efficient updating without dense matrix intermediates.

• J. J. Brust & M. A. Saunders, “Fast and Accurate SVD-Type Updating in Streaming Data,” SIAM J. Math. Data Sci., 2025 (Brust et al., 2 Sep 2025).