Kronecker-CP Decomposition Overview

• Kronecker-CP decomposition is a family of tensor and operator decompositions that combine Kronecker product structure with the canonical polyadic framework for efficient, structured approximations.
• It employs algorithmic steps like reshaping, permutation, CP decomposition via SVD or ALS, and factor recovery to ensure robust, symmetry-preserving representations.
• Applications include neural network compression, structured preconditioning in PDEs, and multiresolution analysis, offering computation-efficient and data-sparse models.

The Kronecker-CP decomposition is a family of tensor and operator decompositions that integrate the structure-inducing capacity of Kronecker products with the canonical polyadic (CP) decomposition framework. Its variants, including the Tensor Kronecker Product SVD (TKPSVD) and Kronecker-CP (KCP) decompositions, generalize the Kronecker product beyond matrices to arbitrary-dimensional tensors, allowing a sum-of-products representation with robust structural and computational properties. These decompositions enable data-sparse, symmetry-preserving, and computation-efficient approximations for multiway arrays and operators, with applications spanning machine learning, numerical analysis, and signal processing.

1. Mathematical Foundations of Kronecker-CP Decomposition

Let $$\mathscr A \in \mathbb R^{n_1 \times n_2 \times \cdots \times n_k}$$ be a $k$-way tensor. The tensor Kronecker product generalizes the matrix Kronecker product by pairing indices, resulting in a tensor $\mathscr C = \mathscr A \otimes \mathscr B$ with shape $n_1m_1 \times \dots \times n_km_k$, where each entry is indexed by groups $[i_r, j_r]$ for $r=1, \dots, k$ and computed as

$$\mathscr C_{[i_1 j_1]\, [i_2 j_2]\, \cdots\, [i_k j_k]} = \mathscr A_{i_1\, i_2\, \dots\, i_k}\; \mathscr B_{j_1\, j_2\, \dots\, j_k}.$$

The core Kronecker-CP decomposition, as in the TKPSVD (Batselier et al., 2015), expresses a tensor as a finite sum of tensor Kronecker products: $$\mathscr A = \sum_{j=1}^R \sigma_j\, \mathscr A_j^{(d)} \otimes \cdots \otimes \mathscr A_j^{(1)}$$ with constraints $\| \mathscr A_j^{(i)} \|_F = 1$ and for each mode $r$, $\prod_{i=1}^d n_r^{(i)} = n_r$.

In operator settings, the Kronecker sum decomposition represents a linear map on matrices as $A(X) \approx \sum_{i=1}^r X_i X Y_i^T$ with $A$ encoded as a fourth-order tensor, and the CP structure arises from grouping mode indices (Dressler et al., 2022).

2. Algorithmic Construction and Uniqueness

Computation of the Kronecker-CP decomposition involves multi-step procedures:

1. Reshape and Permute: Given a tensor and target Kronecker degree $d$, reshape into a $(kd)$-way tensor with factorized mode sizes, and permute axes to group indices corresponding to each Kronecker factor.
2. Polyadic Decomposition: Collapse the structure into a $d$-way tensor and compute an orthogonal rank-1 polyadic (CP) decomposition, using SVD (for $d=2$), HOSVD, or tensor-train rank-1 SVD (TTr1SVD).
3. Recover Kronecker Factors: Each outer product term in the polyadic decomposition maps directly to a Kronecker product in the original tensor via reshaping.

The decomposition exists for arbitrary tensors and prescribed Kronecker shapes. Uniqueness holds essentially up to ordering and signs of factors, as in orthogonal CP expansions, provided the singular values are distinct (Batselier et al., 2015). The minimal number of Kronecker terms required is the Kronecker rank.

A universal algorithmic framework—the Monic Decomposition Algorithm (MDA)—and alternating least squares (ALS) methods have been developed for exact and least-squares Kronecker product decomposability for vectors, matrices, and tensors, leveraging projection operators, swap matrices, and suitable permutations (Cheng, 26 Sep 2025). For matrices and hypermatrices, Kronecker decomposability reduces to that of permuted (vectorized) forms.

3. Structure Preservation and General Symmetric Tensors

A distinctive property of Kronecker-CP decompositions is structure inheritance. If the original tensor exhibits a classic symmetry or structured pattern—such as symmetric, persymmetric, centrosymmetric, Toeplitz, or Hankel structure, formalized as invariance under a permutation $P$ on the entries—then its Kronecker-CP factors inherit this structure for all terms with distinct singular values (Batselier et al., 2015).

The general symmetric tensor notion consolidates these symmetries as invariance under a permutation decomposable via a Kronecker product: $P = Q^T (P_d \otimes \cdots \otimes P_1) Q$, where $Q$ is the reshuffling permutation from the decomposition process. Each Kronecker factor then satisfies $$P_i \mathrm{vec}(\mathscr A_j^{(i)}) = \pm\,\mathrm{vec}(\mathscr A_j^{(i)})$$, thus preserving generalized symmetry at each scale.

Standard CP decompositions do not generically preserve such macro-structure in their factors, whereas Kronecker-CP methods guarantee this property by construction due to the underlying algebra (Batselier et al., 2015).

4. Computational Complexity and Numerical Stability

The cost profile of Kronecker-CP decompositions is governed by the polyadic step:

• For $d=2$, SVD on an $N\times M$ matrix costs $\mathcal O(\min\{NM^2, MN^2\})$.
• HOSVD (for $d>2$) requires one SVD per mode, each of size $n_i\times \prod_{j\neq i} n_j$.
• TTr1SVD chains small SVDs with total cost $\sum_{r=1}^{k-1} O(n_r^3 n_{r+1} \cdots n_k)$.

Reshape and permutation steps are computationally negligible. Storage for Kronecker-CP factors is $R \sum_{i=1}^d \prod_{r=1}^k n_r^{(i)}$. For large tensors with low Kronecker rank, this is significantly sublinear in the full tensor size.

All SVD-based steps inherit backward stability, ensuring robust approximations and error certificates for truncated representations (Batselier et al., 2015).

For spectral-norm optimal Kronecker-CP decompositions, alternating semidefinite programming (SDP) strategies are required, with per-step cost $O((mn)^6)$ due to large PSD constraints, thus applicable to moderate-scale problems (Dressler et al., 2022). The alternating block updates use Schur-complement-based LMIs to guarantee convexity, with provable convergence of iterates by biconvexity arguments.

ALS-based Kronecker decompositions for least-squares error can be initialized with MDA, achieving numerically stable and efficient convergence for both matrix and tensor cases (Cheng, 26 Sep 2025).

5. Comparison with Classical CP and Other Tensor Formats

The Kronecker-CP decomposition differs fundamentally from standard CPD, tensor train (TT), block term (BT), tensor ring (TR), and hierarchical Tucker (HT) decompositions:

Format	Parameters (leading order)	Multiply FLOPs	Structure preservation
CP (rank-$R$)	$R\sum_{i=1}^k n_i$	$O(R\,n_1n_2\cdots n_k)$	No symmetry inheritance
TT	$O((d-2)mn r^2)$	$O(d\,\max(m,n)^{d+1} r^2)$	No
BT	$O((d\,mn\,r + r^d)P)$	$O((d\,\max(m,n)^{d+1} + n^d) r^d P)$	No
TR	$O(d(m+n) r^2)$	$O(d(m^d + n^d) r^3)$	No
HT	$O((d-1)r^3 + d\,mn\,r)$	$O((2d-1)\max(m,n)^{d+1} r^{1+\log_2 d})$	No
Kronecker-CP	$O(d(m+n) r K)$	$O( d\,\max(m,n)^{d}(r+r^2)K )$	Yes

Kronecker-CP achieves polynomial storage in the number of modes $d$, while classical Kronecker decompositions are exponential in $d$. For moderate internal rank $r$, Kronecker-CP outperforms all other tensor formats both in storage and contraction cost, especially for large $K$ (the number of Kronecker terms) (Wang et al., 2020).

Additionally, Kronecker-CP supports direct closed-form truncation error estimates, paralleling the SVD: truncating to the first $r$ terms yields relative Frobenius error

$$\frac{ \| \mathscr A - \sum_{j=1}^r \sigma_j\,\mathscr A_j^{(d)} \otimes \cdots \otimes \mathscr A_j^{(1)} \|_F }{\|\mathscr A\|_F} = \frac{ \sqrt{ \sigma_{r+1}^2 + \cdots + \sigma_R^2 } }{ \sqrt{ \sigma_1^2 + \cdots + \sigma_R^2 } }.$$

6. Applications in Scientific Computing and Machine Learning

Kronecker-CP decompositions provide data-sparse representations and are used in applications such as:

• Compression of neural networks: KCP-decomposed RNN weights (notably LSTM input-to-hidden matrices) can attain compression ratios up to $2.8 \times 10^5$ with negligible accuracy loss, outperforming TT, BT, TR, and HT parametrizations in both parameter count and arithmetic complexity (Wang et al., 2020).
• Structured preconditioners: Leading Kronecker terms extracted from TKPSVDs enable efficient application of separable linear preconditioners in Sylvester and Lyapunov operator equations (Batselier et al., 2015).
• Multiresolution analysis: The first few Kronecker factors efficiently encode low resolution approximations of images or multiway signals, offering straightforward control of approximation error.
• Fast convolutions and PDE discretizations: Many multiway convolution operations and grid-based PDE operators reduce to low-Kronecker-rank approximations, facilitating fast algorithms and storage savings.

Parallelization is inherently supported because the summands in Kronecker-CP (e.g., KCP) are independent; modern architectures can dispatch each Kronecker term contraction independently and sum partial results (Wang et al., 2020).

7. Recent General Theory and Universal Solvability

Recent work provides a universal framework for Kronecker product decomposition across vectors, matrices, and tensors, with necessary and sufficient decomposability conditions (Cheng, 26 Sep 2025). The Monic Decomposition Algorithm exploits sparse projections and a head-index paradigm to certify and recover exact Kronecker decompositions with complexity $O(kN)$ for vectors of size $N = \prod_{i=1}^k n_i$. Swap and permutation matrices reduce higher-order KPDs to vectorized forms, and alternating least squares enables least-squares and finite-sum Kronecker decompositions with efficient convergence and practical error control.

This unifying theory confirms that all KPD questions—exactness, approximation, multi-term expansions—admit algorithmic solutions with polynomial complexity for structured tensors, generalizing and operationalizing Kronecker-CP decompositions in applied and theoretical contexts (Cheng, 26 Sep 2025).