Shift and Scale Invariance

• Shift and scale invariance are transformation symmetries defined by invariance under translations and uniform dilations, central to physical laws, probability distributions, and learning systems.
• They dictate the form of key distributions such as exponential, Gaussian, and power-law types, ensuring robustness and universality in statistical mechanics and critical phenomena.
• Practical applications include CNN architectures that aggregate multi-scale features and quantum field models where anomalous breaking yields measurable shifts in system dynamics.

Shift and scale invariance are foundational transformation symmetries appearing across mathematical physics, probability theory, quantum field theory, statistical mechanics, and deep learning. Shift invariance (invariance under translation) and scale invariance (invariance under uniform dilations) severely constrain the functional forms of physical laws, probability distributions, field theories, and feature-extraction architectures. Their interplay is central to universality, critical phenomena, and the robustness of statistical and learning systems.

1. Definitions and Formal Properties

Let $x\in\mathbb{R}^n$, $T_a: x\mapsto x+a$ denote translation, and $S_b: x\mapsto b\,x$, $b>0$, denote uniform scaling (dilation). A function or operator $F$ is shift-invariant if $F(T_a[x]) = F(x)$ for all $a$. It is scale-invariant if $F(S_b[x]) = F(x)$ for all $b$ in the domain. For probability densities $p(x)$, shift and stretch invariance impose

$$p(x+a) \propto p(x) \quad \forall\, a \in \mathbb{R}^n,\qquad p(b\,x) \propto p(x) \quad \forall\, b>0.$$

In machine learning, for an image $f:\mathbb{R}^2\to \mathbb{R}$ and a feature extractor $\Phi$, shift invariance and scale invariance respectively require $\Phi[f(x+\tau)] = \Phi[f(x)]$ and $\Phi[f(a\,x)] = \Phi[f(x)]$ for all admissible $\tau$, $a$ (Jansson et al., 2021).

2. Probability Laws and Statistical Patterns from Invariance

Shift invariance alone imposes strong constraints on admissible probability distributions. For a real variable $x$ and density $p(x)$:

• If $p(x)$ is shift-invariant, a functional equation yields $p(x) = C e^{-\lambda x}$, i.e., the exponential law (Frank, 2016).
• In $n$-dimensions, imposing rotational and stretch invariance on $p(x)$ leads uniquely to the Gaussian family: $$p(x) = \Bigl(\tfrac{\lambda}{\pi}\Bigr)^{n/2} e^{-\lambda \|x\|^2}$$ (for some $\lambda>0$) (Frank, 2016).
• Applying invariance in alternative (homogeneous) scales $T(x)$, power-law (Pareto), stretched-exponential (Weibull), gamma, and Student distributions appear as the only permissible classes. This approach is structurally distinct from (and more general than) entropy maximization or canonical ensemble postulates, as it directly ties observed scaling relations to underlying symmetry (Frank, 2016).

3. Spontaneously Broken Scale Invariance, Shift Symmetry, and Cosmology

In certain quantum field theories, notably inflationary cosmology, global scale invariance leads to a Goldstone boson (dilaton) whose low-energy effective action exhibits a shift symmetry. For a Jordan-frame Lagrangian featuring scale invariance, Weyl rescaling yields an Einstein-frame action where the field $\varphi$ enjoys an exact shift symmetry $\varphi \to \varphi + \mathrm{const}$: $$\mathcal{L}_E = \sqrt{-g}\left[ \tfrac{M_{\rm Pl}^2}{2} R - \tfrac12 (\nabla\varphi)^2 - V(\varphi) \right]$$ with a flat potential $V(\varphi) = \mathrm{constant}$ for exact scale invariance (Csaki et al., 2014). Slight explicit breaking from nearly-marginal operators introduces a small parameter $\epsilon$, generating a non-trivial potential while preserving the radiative stability and naturally allowing large field excursions. This structure guarantees both slow-roll protection and a near scale-invariant spectrum of primordial perturbations, as required by CMB data (Csaki et al., 2014).

4. Shift and Scale Invariance in Quantum Field Theory and Critical Phenomena

Shift symmetry and scale invariance jointly constrain admissible field-theoretic interactions. In scalar field theories with $2k$-derivative kinetic terms and marginal shift-symmetric interactions, two infinite families arise:

• Quartic: $$V_{\rm int}^{(4)} = \frac{g}{4}(\partial\phi\cdot\partial\phi)^2$$
• Cubic: $$V_{\rm int}^{(3)} = \frac{g}{2}(\partial\phi\cdot\partial\phi)\Box\phi$$ Both exhibit shift symmetry, forbidding relevant deformations that could spoil conformal fixed points. At criticality obtained via RG analysis ($\varepsilon$-expansion near the upper critical dimension), these theories are not only scale- but also conformally-invariant at the one-loop fixed point, admitting a conserved, traceless improved energy–momentum tensor. The shift symmetry is essential to the stability of these non-unitary scalar CFTs and their fixed-point structure (Safari et al., 2021).

5. Anomalous Breaking of Scale Invariance: Quantum Anomaly

In two-dimensional few-body and many-body physics, classical scale invariance present in Hamiltonians (e.g., for a harmonically trapped Fermi gas) can be explicitly broken at the quantum level. Renormalization introduces a length scale (the 2D scattering length $a_{2D}$), breaking continuous scale invariance and yielding a quantum anomaly. In the 2D Fermi gas, this anomaly manifests as a shift in the breathing mode frequency: $$\omega_B = 2\omega_0 \to \omega_B = 2\omega_0 + \delta\omega_B$$ with $\delta\omega_B > 0$ near resonance, as verified experimentally. The shift vanishes in both BEC and BCS limits and scales with interaction and temperature-dependent parameters (e.g., the Tan contact) (Holten et al., 2018). Measurement of this frequency shift provides a direct probe of scale symmetry breaking beyond classical expectations.

6. Shift and Scale Invariance in Deep Learning Architectures

Invariance principles are operationalized in convolutional neural networks (CNNs) via structural design. Ordinary convolution and pooling layers yield translation invariance, but scale invariance is not automatically guaranteed. To achieve both:

• Scale-channel networks process input at multiple sampled scales. With weight sharing and pooling (max or average) over scale channels, the resulting architectures (FovMax, FovAvg) achieve provable invariance to both scale and translations.
• For an image $f$ and base CNN $\phi$, foveated scale-channel architectures construct feature maps $\phi_{a_i}[f]$ at rescaled supports, aggregating by max or mean pooling over $a_i$: $$\Phi_{\max}[f](x,c) = \max_i \left\{ \phi_{a_i}[f](x, c) \right\}$$

$$\Phi_{\rm avg}[f](x, c) = \frac{1}{N}\sum_i \phi_{a_i}[f](x, c)$$

• Empirically, such architectures generalize perfectly to unseen scales provided scale channels densely cover the operative range, and translation invariance is inherited via the convolutional backbone (Jansson et al., 2021).

7. Physical and Applied Significance

Shift and scale invariance structure the fundamental patterns of statistical mechanics, critical dynamics, field-theoretic RG flows, and learning systems. Their explicit realization yields universal distribution forms and assures robustness against parameterization and data variation. When symmetry is anomalously broken (e.g., by regulator- or interaction-induced length scales), new physical phenomena such as quantum anomalies or explicit mass gap generation arise (Holten et al., 2018, Csaki et al., 2014). In deep learning, embedding both invariances at the architectural level enhances generalization across domains characterized by unknown scale and translation variability (Jansson et al., 2021). In all cases, it is the interplay of symmetry, its breaking, and the resultant invariance properties that governs the universality and predictivity of the system.