Iterative Alternating Matrix-Scaling

• The paper introduces an alternating minimization process that scales matrix rows and columns to meet prescribed marginals, with convergence guaranteed for strictly positive matrices.
• It employs convex optimization and information-geometric projections to minimize divergence measures, leading to rapid, quantifiable convergence under specific conditions.
• The framework generalizes to operator and tensor scaling, enabling practical applications in quantum information processing, numerical preconditioning, and combinatorial optimization.

The Iterative Alternating Matrix-Scaling Algorithm encompasses a class of alternating minimization procedures designed to scale the rows and columns of a matrix (or, more generally, a tensor or operator) to achieve prescribed marginals. The classical instance is the Sinkhorn–Knopp algorithm, which, through iterative application of diagonal scalings, converges to a doubly stochastic (row- and column-sum one) matrix or certifies non-scalability. This framework is tightly interwoven with convex optimization, information geometry, combinatorics (notably perfect matching theory), invariant theory, and quantum information processing.

1. Mathematical Formulation and Algorithmic Structure

Given $A \in \mathbb{R}^{n\times n}_+$ with $A_{ij} \geq 0$, and positive target row and column marginals $r \in \mathbb{R}^n_+$, $c \in \mathbb{R}^n_+$ (satisfying $\sum_i r_i = \sum_j c_j$), the goal is to find diagonal matrices $X = \operatorname{diag}(x)$, $Y = \operatorname{diag}(y)$, $x,y > 0$, so that $M = XAY$ satisfies $(M\mathbf{1})=r$, $A_{ij} \geq 0$0. In the special case $A_{ij} \geq 0$1, $A_{ij} \geq 0$2 is doubly stochastic. The generic iterative scheme alternates "row-scaling" (normalizing each row to the required row sum) and "column-scaling" (each column to the required column sum):

$c \in \mathbb{R}^n_+$1

Each row/column-normalization step constitutes an I-projection or e-projection in the language of information geometry (Matsuda et al., 2020).

2. Convergence, Complexity, and Theoretical Guarantees

The foundational Sinkhorn–Knopp theorem asserts that if $A_{ij} \geq 0$3 is strictly positive, the alternated scaling converges to a unique solution, up to diagonal scaling symmetry, yielding $A_{ij} \geq 0$4 exactly doubly stochastic (Nathanson, 2019, Chakrabarty et al., 2018). The convergence is geometric, with explicit iteration bounds:

• For positive matrices, $A_{ij} \geq 0$5-error $A_{ij} \geq 0$6 is achieved in $A_{ij} \geq 0$7 iterations (Chakrabarty et al., 2018).
• For nonnegative matrices, convergence occurs if and only if the support graph has a perfect matching (Hayashi et al., 2022).

Recent work identifies a sharp phase transition: for normalized matrices with density $A_{ij} \geq 0$8, the algorithm converges in $A_{ij} \geq 0$9 iterations; if $r \in \mathbb{R}^n_+$0, lower bounds of $r \in \mathbb{R}^n_+$1 (in $r \in \mathbb{R}^n_+$2-error) apply, and no polynomial-time guarantee exists (He, 13 Jul 2025).

Crucially, the algorithm is in parallel complexity class NC and can be implemented efficiently on parallel architectures (Chakrabarty et al., 2018, Garg et al., 2018).

3. Optimization, Invariant Theory, and Information-Geometric Interpretations

Alternating matrix-scaling is formally an alternating minimization procedure for convex objectives:

• KL-divergence viewpoint: Each iteration alternately minimizes Kullback–Leibler divergence to the set of matrices with prescribed row (or column) sums (Chakrabarty et al., 2018, Matsuda et al., 2020).
• Potential functions: Convergence is tracked via decreases in a convex potential, e.g., negative entropy or matrix permanent-based functionals (Garg et al., 2018).
• Invariant theory: The scalable matrices correspond to those with nonzero capacity under group action ($r \in \mathbb{R}^n_+$3), characterized via the null-cone problem (Bürgisser et al., 2017, Garg et al., 2018).
• Information geometry: Each normalization is an e-projection onto a marginal-constraint submanifold in the information-geometric (Fisher) metric; monotone decrease of KL-divergence guarantees convergence (Matsuda et al., 2020).

4. Structural, Combinatorial, and Non-Scalable Cases

If $r \in \mathbb{R}^n_+$4 cannot be scaled to the prescribed marginals (i.e., no perfect matching in the bipartite support graph), the alternating process produces at most two limit points with possibly block-diagonal structure (Aas, 2012). Recent advances provide efficient algorithms for decomposing $r \in \mathbb{R}^n_+$5 into its support blocks and explicitly describing these limits (Aas, 2012, Hayashi et al., 2022). The non-scalability is detected by identification of Hall blockers—subsets violating Hall's condition (Hayashi et al., 2022).

Key combinatorial and geometric programming interpretations relate the feasibility of scaling to stable-set inequalities and the Dulmage-Mendelsohn decomposition of bipartite graphs.

5. Generalizations: Operator and Tensor Scaling

Scaling extends naturally to operators and tensors, crucial in quantum information and invariant theory (Bürgisser et al., 2017, Matsuda et al., 2020):

• Operator Scaling: Given a completely positive map $r \in \mathbb{R}^n_+$6 represented by its Choi matrix, the operator Sinkhorn procedure alternately normalizes the left and right quantum marginals via matrix square roots and inverses, converging to a doubly stochastic map under rank nondeficiency (Matsuda et al., 2020).
• Tensor Scaling: The optimization generalizes to multilinear actions on higher-order tensors; the objective is minimizing the norm under $r \in \mathbb{R}^n_+$7 action. Alternating minimization is performed block-wise over tensor modes, with polynomial-time convergence to $r \in \mathbb{R}^n_+$8-approximation (Bürgisser et al., 2017).

6. Numerical Examples, Finite-Step Phenomena, and Explicit Closed-Form Limits

For certain low-dimensional or highly symmetric cases, the Sinkhorn process can terminate in finitely many steps, or explicit algebraic formulas for the limit can be derived (Nathanson, 2019, Nathanson, 2019, Nathanson, 2019). For example, some families of $r \in \mathbb{R}^n_+$9 singular matrices become doubly stochastic after a single column scaling; in the generic case, convergence is exponential but not finite except in degenerate situations (Nathanson, 2019).

Closed-form limits have been obtained for all symmetric positive $c \in \mathbb{R}^n_+$0 matrices with two distinct entries, exhibiting algebraic dependencies of degrees up to eight (Nathanson, 2019, Nathanson, 2019). Iterates provide rational approximations to these algebraic numbers, linking matrix-scaling processes to Diophantine approximation.

7. Applications and Extensions

Alternating matrix-scaling underlies numerous algorithmic and structural applications:

• Approximate computation of matrix permanents and thus intractable #P-complete combinatorial quantities (Garg et al., 2018).
• Statistical fitting and maximum-entropy couplings (iterative proportional fitting).
• Design-matrix rank lower bounds in combinatorial geometry.
• Preconditioning linear systems in numerical linear algebra.
• Quantum circuit synthesis and operator decomposition; for unitaries, a related alternating phase-normalization yields line-sum-unitary matrices suitable for efficient quantum circuit decomposition (Vos et al., 2014).

Further generalizations to block-scaling, weighted marginals, and accelerated algorithms have been developed, with nearly linear-time performance using combinatorial and convex optimization techniques (Garg et al., 2018).

---

References:

• (Aas, 2012) Limit points of the iterative scaling procedure
• (Vos et al., 2014) Scaling a unitary matrix
• (Bürgisser et al., 2017) Alternating minimization, scaling algorithms, and the null-cone problem from invariant theory
• (Chakrabarty et al., 2018) Better and Simpler Error Analysis of the Sinkhorn-Knopp Algorithm for Matrix Scaling
• (Garg et al., 2018) Recent progress on scaling algorithms and applications
• (Nathanson, 2019) Matrix scaling, explicit Sinkhorn limits, and arithmetic
• (Nathanson, 2019) Matrix scaling limits in finitely many iterations
• (Nathanson, 2019) Matrix scaling and explicit doubly stochastic limits
• (Matsuda et al., 2020) Information geometry of operator scaling
• (Hayashi et al., 2022) Finding Hall blockers by matrix scaling
• (He, 13 Jul 2025) Phase transition of the Sinkhorn-Knopp algorithm