Invariant Divergence Overview

• Invariant divergence is a class of divergence measures that remain unchanged under specific transformations such as scaling, reparameterization, and group actions.
• It is pivotal in fields like probability, convex geometry, quantum information, and robust hypothesis testing by preserving key statistical and structural properties.
• These methods empower reliable inference and classification by eliminating artifacts from coordinate choices, normalization, and arbitrary labeling.

Invariant divergence refers to a class of divergence measures and related mathematical phenomena that are unchanged under certain natural symmetries, scalings, transformations, or reparameterizations. This invariance is critical in probability, statistics, convex geometry, risk theory, quantum information, variational inference, operator theory, group theory, and physics. The concept appears in diverse forms, including divergences invariant under scaling (e.g., scale-invariant divergences), group actions (e.g., permutations or linear transformations), probabilistic reference changes, coordinate transformations, and more. Invariant divergences are central to robust statistical procedures, the analysis of physical systems, and the classification of mathematical structures.

1. Formal Definition and Core Examples

A divergence is typically a functional $D(P\|Q)$ or $D(p,q)$ that quantifies the "distance," asymmetry, or distinguishability between two mathematical objects—often probability distributions, measures, density functions, vector fields, or geometric bodies.

Invariance means $D$ satisfies a symmetry property with respect to a group of transformations $\mathcal{G}$: $$D(\tau(P),\tau(Q)) = D(P,Q), \quad \forall\tau\in\mathcal{G}.$$

Key instances:

• Permutation invariance (statistical hypothesis testing): A divergence $D$ between categorical distributions $P,Q$ is invariant if $D(P\circ T^{-1}, Q\circ T^{-1})=D(P,Q)$ for any symbol permutation $T$ (Harsha et al., 2024).
• Scale invariance (variational inference): $D(cp, cq) = D(p, q)$ for all $c>0$, e.g., scale-invariant Alpha-Beta divergence (Regli et al., 2018).
• SL(n)-invariance (convex geometry): For convex bodies $K\subset\mathbb{R}^n$, $D_f(P_{T(K)}, Q_{T(K)}) = D_f(P_K, Q_K)$ for all $T \in \mathrm{SL}(n)$ (Werner, 2012).
• Reparameterization invariance (optimal transport): $$D_{T, f}(p\|q) = D_{\tilde T, f}(\tilde p\|\tilde q)$$ for any smooth diffeomorphism $k$ (Li, 22 Apr 2025).
• Gauge invariance in field theory: Physical observable's diverging terms cancel when constructed to be invariant under gauge transformations (Urakawa et al., 2010).
• Local isometric invariance (quantum information): Divergence-based quantum information measures are invariant under local isometries or unitaries (Popp et al., 4 Sep 2025).

2. Invariant Divergence in Statistical Inference and Hypothesis Testing

Invariant divergences play a central role in robust statistical prediction and testing. In composite hypothesis tests, invariance ensures the procedure's performance is independent of arbitrary labellings or coordinate choices.

• Definition (discrete, finite alphabet): For $P,Q$ with full support, $D$ is invariant if it is unchanged under relabeling of the outcome space (Harsha et al., 2024).
• Examples: Classical $f$-divergences (Kullback-Leibler, $\chi^2$, Rényi) are permutation-invariant; so are the quadratic forms tied to Fisher information.
• Second-order optimality: Invariant divergence tests exhibit a universal second-order loss due to $\chi^2$-quantile dependence. Non-invariant divergences can be tailored and may outperform invariant ones against restricted alternatives (Harsha et al., 2024).
• Robustness property: Invariance rules out the introduction of artifacts related to the order or basis in both parametric and nonparametric settings.

3. Scale, Reference, and Reparameterization Invariance

A divergence is scale-invariant if, up to normalization, it depends only on the relative, not absolute, magnitudes: $$D(c_1 p \| c_2 q) = D(p \| q), \quad \forall c_1, c_2 > 0.$$ This is critical in Bayesian inference, information geometry, and the design of reference-invariant health disparity measures (Regli et al., 2018, Talih, 2013).

Reference-invariant indices (e.g., Rényi-based RI/SRI):

• Do not depend on the specific reference subgroup or population mean.
• Extend or limit to classical measures: mean log deviation, Theil, Atkinson indices (Talih, 2013).
• Are robust to statistical noise in subgroup identification, improving reliability in real data applications.

Reparameterization invariance, as in transport $f$-divergences, delivers full invariance under smooth coordinate changes: $$D_{T,f}(p\|q) = D_{\tilde T, f}(\tilde p \| \tilde q),$$ where $k$ is any smooth bijection and $\tilde q = (k^{-1})_\# q$ (Li, 22 Apr 2025).

4. Invariants in Geometry, Group Theory, and Operator Theory

Geometric and group-theoretic invariants:

• f-divergence for convex bodies: $D_f(P_K\|Q_K)$ is SL(n)-invariant over linear transformations, acting as a valuation on the union/intersection of convex bodies (Werner, 2012).
• Divergence in geometric group theory: The divergence function, detour length, and divergence spectrum are invariants under quasi-isometry, classifying group growth types (polynomial, exponential) and distinguishing between subgroup structures (Goldsborough et al., 2023, Tran, 2016).
• Right-inverse of curl (operator theory): Constructed to be invariant under divergence-free subspaces, yielding solutions to PDEs (e.g., Helmholtz decomposition, Maxwell, Beltrami, Vekua problems) with invariance under boundary conditions (Delgado et al., 2023).
• Divergence-form operators (PDEs): Existence and uniqueness of invariant measures are tied to symmetry and (anti-)symmetry in the coefficients; invariance is essential for uniqueness/ergodicity (Lee, 2021).

Table: Invariant Divergences and Principal Domains

Domain	Invariance Structure	Key Example
Statistics	Permutation/Scale	f-divergences, scale-invariant AB
Convex Geometry	SL(n) linear action	Convex body f-divergence
Quantum Info	Local isometries/unitary	Divergence-based mutual information
Optimal Transport	Diffeomorphism/Pushforward	Transport f-divergence
Group Theory	Quasi-isometry	Divergence spectrum, random divergence
PDE/Operators	Divergence-free/Gauge	Right-inverse curl, invariance of L
Physics/QFT	Lorentz invariance	Causal spectral divergences

5. Robustness, Reference-Invariance, and Applications

Invariant divergences are preferred for the following reasons:

• Statistical robustness: Tests or indices remain valid regardless of arbitrary choices of reference group or data normalization. For instance, the Rényi-based RI/SRI health indices are robust to subgroup selection and provide well-behaved inference under various survey designs (Talih, 2013).
• Physical interpretation: In quantum field theory, divergences in perturbation theory can be "gauge artifacts" and vanish in truly gauge-invariant observables (Urakawa et al., 2010). Lorentz-invariant divergences admit physically meaningful, convergent spectral decompositions (Mashford, 2018).
• Geometry and group theory: Quasi-isometry and SL(n) invariance ensure that invariants classify spaces up to large-scale geometric or algebraic deformations, underpinning modern rigidity and classification results (Tran, 2016, Goldsborough et al., 2023).

6. Spectral and Dynamical Notions of Invariant Divergence

In ergodic theory, time-invariant or gauge-invariant divergences characterize rates and limiting behaviors:

• Hidden Markov Models: Asymptotic rates of Rényi divergence for HMMs are characterized via the top Lyapunov exponent of a matrix cocycle, independent of initial distributions—an invariance tied to the existence of an invariant measure for the underlying process (Fuh et al., 2021).
• Quantum systems: Quantum divergences are invariant under local isometries/unitaries, as shown via data-processing and reversal channel constructions (Popp et al., 4 Sep 2025).
• Field theory: Lorentz-invariant measures admit spectral decompositions parameterized only by the mass shell, and physical quantities ("vacuum polarization functions," running coupling constants) constructed this way are free of ultraviolet divergences by design (Mashford, 2018).

7. Foundational Role and Open Directions

Invariant divergences unify a broad spectrum of concepts, from abstract measure theory and information geometry to concrete physical and statistical systems. Open directions include:

• Characterization of optimal non-invariant divergences for composite testing (Harsha et al., 2024).
• Computation and algorithmic classification of divergence spectra for wide classes of groups (Tran, 2016).
• Deeper geometric understanding of transport-based and mapping-based divergence invariances (Li, 22 Apr 2025).
• Extension to high-dimensional, continuous, or nonparametric domains, where invariance is necessary for universality and replicability.

Invariant divergence thus serves as a central organizing principle for constructing and analyzing functionals that reliably compare, discriminate, or classify probability distributions, geometric objects, operators, and quantum or dynamical systems—free from the ambiguities and artifacts induced by the choice of coordinates, labeling, or reference.