Matrix-Tensor Congruence

• Matrix-tensor congruence is defined as the equivalence of matrices, matrix tuples, and tensors via simultaneous change of basis, uniting concepts from linear and multilinear algebra.
• It generalizes classical congruence results to symmetric and skew-symmetric settings, ensuring that equivalence implies congruence for structured algebraic objects.
• The framework supports efficient data modeling in statistics by matching the true degrees of freedom with tensor modes, thereby reducing computational overhead and storage.

Matrix-tensor congruence concerns the precise relationship between linear and multilinear algebraic objects—specifically, the equivalence and classification of spaces of matrices, tuples of matrices, and higher-order tensors under congruence group actions. This subject unites structural linear algebra, multilinear algebra, and computational statistics, and has ramifications for problems ranging from multilinear map classification to tensor-structured statistical modeling and path identifiability in stochastic analysis.

1. Matrix and Tensor Congruence: Definitions and Group Actions

Matrix congruence is defined for spaces $V,W\subset\mathbb{C}^{n\times n}$ as the existence of an invertible $S$ such that $S V S^{\top} = W$. More generally, two matrices $A,B$ are congruent if $B = S A S^{\top}$ for some $S \in \mathrm{GL}_n(\mathbb{C})$. For matrix tuples $(A_1,\ldots,A_t)$, congruence is defined analogously: the tuple $(B_1,\ldots,B_t)$ is congruent to $(A_1,\ldots,A_t)$ if $B_i = S A_i S^{\top}$ for all $i$.

Tensor congruence generalizes this notion to $k$-mode arrays: for tensors $T\in(K^m)^{\otimes k}$ over a field $K$, the group $\mathrm{GL}(m,K)$ acts via

$$(X\cdot T)_{i_1\ldots i_k} = \sum_{\alpha_1,\ldots,\alpha_k=1}^m t_{\alpha_1\ldots\alpha_k}\,x_{i_1\alpha_1}\cdots x_{i_k\alpha_k}\,.$$

This action is the simultaneous change of basis in every mode, establishing orbits whose structure is fundamental in the classification theory of tensors and multilinear maps (Belitskii et al., 2020, Pfeffer et al., 2018).

2. Equivalence and Congruence for Matrix Spaces, Tuples, and Multilinear Maps

For two linear subspaces of matrices $V,W\subset\mathbb{C}^{n\times n}$, equivalence denotes the existence of invertible $S,R$ such that $S V R = W$, i.e., $B = S A R$ for $A \in V, B \in W$. Congruence is the restriction to $R = S^{\top}$. A central result is that for spaces of symmetric or skew-symmetric matrices, equivalence and congruence coincide: $V$ and $W$ are congruent if and only if they are equivalent.

This result extends to tuples: if $(A_1,\ldots,A_t)$ and $(B_1,\ldots,B_t)$ are tuples of symmetric or skew-symmetric matrices, then ordinary equivalence implies congruence [(Belitskii et al., 2020), Theorem 1(a)]. This principle further extends to $k$-linear maps and tensors. For multilinear maps $F:U^k\to V$ and $G:(U')^k\to V'$, equivalence (via possibly different invertible $\varphi_1,\ldots,\varphi_k$ for each slot) collapses to congruence (a single invertible $\varphi$ for all slots) in the symmetric and skew-symmetric setting for $k\geq 2$, i.e., symmetric-equivalence implies congruence [(Belitskii et al., 2020), Theorem 2(a)].

These phenomena are underpinned by algebraic arguments involving self-adjoint operators, polynomial functional calculus (Gantmacher’s Lemma for matrices, eigen-decomposition for tensors), and reduction-theoretic approaches via quiver representations.

3. Matrix–Tensor Congruence in Statistical Models

Matrix–tensor congruence in statistical modeling, particularly in the General Linear Model (GLM), is the situation where the rank of the array representing data exactly matches the true degrees of freedom (DoF) in the multivariate structure (Kress, 2023). In the classical matrix-based GLM, complex designs (e.g., group-by-regressor-by-observation) require flattening and block-padded matrices that introduce an incongruence: the data structure’s rank and shape over-represent the true underlying DoF, leading to superfluous zero blocks and excess memory usage.

The tensor formulation writes the model directly as

$$\mathcal{Y}_{i_1 i_2 \dots i_K} = \mathcal{X}_{i_1 i_2 \dots i_K j}\,\mathcal{B}_j + \mathcal{E}_{i_1 i_2 \dots i_K}$$

with each mode of the tensor $\mathcal{X}$ corresponding precisely to an independent data variation (e.g., group, subject, regressor), achieving matrix–tensor congruence. This reformulation improves memory efficiency and computation by avoiding the construction and manipulation of large, sparse block matrices. The resulting savings in storage are substantial—by a factor of $n + n/m$ for $n$ groups and $m$ regressors—when compared to the classical, incongruent matrix approach (Kress, 2023).

4. Stabilizers, Orbits, and Identifiability in Tensor Congruence

The stabilizer of a tensor $T$ under congruence, defined as $$\mathrm{Stab}_K(T) = \{X \in \mathrm{GL}(m,K): X\cdot T = T\}$$, plays a crucial role in identifiability. A tensor is symmetrically concise if no proper subspace $W\subsetneq K^m$ exists with $T\in W^{\otimes k}$; this is equivalent to the principal flattenings of $T$ having full rank. For concise tensors, the stabilizer generically reduces to the scalar $k$-th roots of unity $\{\eta I : \eta^k = 1\}$, implying identifiability up to these trivial symmetries [(Pfeffer et al., 2018), Theorem 4.1].

For instance, in the recovery of a matrix $X$ from a third-order signature tensor $S = [\![C;X,X,X]\!]$ with a generic or piecewise-linear core tensor $C$, $X$ is uniquely determined up to cube roots of unity if $C$ has trivial stabilizer. The dimension of the universal signature variety $\mathcal{U}_{m,3}\subset(\mathbb{C}^m)^{\otimes 3}$ governs the identifiability threshold, matching the number of parameters $md$ when $m \leq (1/3)d^2 + (1/2)d + (1/6)$ (Pfeffer et al., 2018).

5. Computation and Numerical Aspects

Congruence problems, particularly in the tensor setting, require efficient computational methods. Symbolic approaches solve the polynomial congruence equations via Gröbner basis methods, practical for small dimensions only. Numerical optimization is the technique of choice for higher-dimensional settings: the cost function $g(X) = \|[\![C;X,X,X]\!] - S\|^2$ is minimized by alternating strategies—quasi-Newton (BFGS) for global progress and trust-region Newton methods for refinement—often implemented in MATLAB via Manopt or Tensorlab (Pfeffer et al., 2018).

Numerical non-identifiability is quantified by $$\kappa(X,C) = (\|X\|^3\|C\|) / \inf_{Y:\mathrm{rank}(Y)<m}\|[\![C;X,X,X]\!] - [\![C;Y,Y,Y]\!]\|$$, with condition bounds determined by the minimal singular values of the core tensor’s flattenings. Certain special tensors like the axis-aligned $C_{axis}$ have uniformly bounded $\kappa$, while others (e.g., monomial tensors) exhibit exponential growth with dimension, reflecting numerical instability [(Pfeffer et al., 2018), Corollary 7.3].

For underdetermined settings ($m \gg d$), the set of congruence solutions is large; one selects canonical elements, e.g., the shortest $X$ by imposing a length regularization, minimizing $\ell(X) + \lambda g(X)$.

6. Broader Implications and Classification Results

The “equivalence implies congruence” theorems extend classical results (e.g., Mal'cev’s for symmetric pencils) to the broad setting of arbitrary subspaces of symmetric/skew-symmetric matrices, matrix tuples, and $k$-linear forms. This yields a uniform approach to classifying bilinear and multilinear forms, tensor classification up to order three, and linear operators preserving specified forms, as in Sergeichuk’s quiver theory (Belitskii et al., 2020).

Matrix–tensor congruence provides a foundation for efficient and principled data structure design in computational statistics, ensuring correspondence between mathematical DoF and container rank, with benefits including reduction in storage, computational savings, and enhanced adaptivity for structured data (Kress, 2023). In stochastic analysis and path reconstruction, tensor congruence underpins identifiability and recovery methods based on signature tensors, combining algebraic geometry, representation theory, and numerical analysis (Pfeffer et al., 2018).

7. Illustrative Examples and Applications

Examples provided include:

• Two-group, one-regressor regression in tensor GLM formulation, demonstrating elimination of superfluous zero blocks and exact matching of data modes to DoF (Kress, 2023).
• Recovery of paths from third-order signature tensors, where identifiability is established for piecewise-linear and generic core tensors, and explicit algorithms recover the congruence matrix via numerical minimization (Pfeffer et al., 2018).
• Path reconstruction with minimal length among congruent solutions, obtained by combining objective regularization and congruence constraints.

The algebraic geometry of the universal signature variety, stabilizer classification theorems, and explicit computational strategies coalesce to elucidate the structure and practical significance of matrix-tensor congruence in modern mathematical and statistical applications (Belitskii et al., 2020, Kress, 2023, Pfeffer et al., 2018).