Critical Memory Exponent

• Critical Memory Exponent is a scaling parameter that distinguishes between short-term (Markovian) dynamics and long-memory behavior across systems.
• It plays a key role in non-Debye relaxation, anomalous diffusion, and modified critical phenomena by setting thresholds for distinct dynamical regimes.
• Its precise value determines system responses in nonlinear PDEs, neural network storage, and optimization algorithms, influencing stability and convergence.

A critical memory exponent is a model parameter or scaling exponent that governs the transition between qualitatively distinct dynamical or statistical regimes in systems with temporally nonlocal (“memory”) interactions. These exponents regulate the nature of relaxation, scaling, storage, criticality, and universality in a wide array of contexts including anomalous transport, non-Debye relaxation, critical phenomena with memory, nonlinear wave propagation, optimization algorithms with memory, and high-capacity memory in neural networks. The value of the critical memory exponent typically separates regimes of standard “Markovian” or short-memory dynamics from those with extended, scale-free, or anomalous memory effects, with precise consequences for scaling laws, phase transitions, and functional capacity.

1. Definitions and Representative Models

Critical memory exponents appear in diverse mathematical settings:

• Laplace (or Lévy) Exponent in Stochastic Subordination: For non-Debye relaxations and anomalous diffusion, the characteristic exponent $\alpha$ arises from the Laplace exponent $\phi(s)$ of a non-decreasing infinitely divisible process, with long-memory entering through $\phi(s)\sim s^{\rho}$ for small or large $s$ (Górska et al., 2021).
• Power-Law Memory Kernel Exponents: In critical phenomena and field theories with memory, a power-law temporal kernel $(t-t')^{-\theta}$ introduces a memory exponent $\theta$ governing temporal correlations (Zeng et al., 2022).
• Critical Exponents in Wave Equations: For damped wave equations with nonlinear memory, the parameter $\gamma$ in memory kernels $(t-s)^{-\gamma}$ sets a threshold $p_c(n,\gamma)$ for the existence/blow-up of solutions (Fino, 2010).
• Storage Exponents in Neural Networks: In exponential Hopfield models, the memory exponent $\theta$ regulates the exponential scaling of storage capacity: $P=\exp(\theta N)$. The critical value $\theta_c$ marks a sharp phase transition in retrieval performance (Albanese et al., 8 Sep 2025).
• Optimization Algorithms with Memory: In SGD or GD algorithms with memory, the memory-related schedule exponent $\bar{\alpha}$ controls acceleration of convergence rates and stability boundaries (Yarotsky et al., 2024).

The critical memory exponent is thus not a universal constant but a context-dependent threshold parameter that captures the onset and scaling of memory-induced modifications to the base dynamics.

2. Critical Memory Exponents in Relaxation and Anomalous Diffusion

In non-Debye relaxation models (e.g., dielectric response in complex materials), the governing equations are cast in terms of convolutions with memory kernels or, equivalently, time-smeared derivatives. These are described by a Laplace exponent $\phi(s)$, whose small- or large-$s$ behavior is controlled by the characteristic exponent $\alpha$ (Górska et al., 2021):

• For Debye (Markovian) relaxation, $\alpha=1$ and $\phi(s)\sim s$; memory kernels collapse to delta functions, yielding purely exponential relaxation.
• For non-Debye (fractional, long-memory) relaxation, $0<\alpha<1$, and $\phi(s)\sim s^\alpha$ or similar. The associated memory kernels develop heavy power-law tails.

The transition between these regimes is governed by the critical memory exponent $\alpha_c=1$. At $\alpha<1$, relaxation tails become algebraic ($n(t)\sim t^{-\alpha}/\Gamma(1-\alpha)$), the spectral response shows a high-frequency “excess wing” scaling as $\omega^{-\alpha}$, and, in diffusion problems, the mean-squared displacement exhibits anomalous diffusion ($\langle x^2(t)\rangle\sim t^{\alpha}$) (Górska et al., 2021).

3. Critical Memory Exponents in Phase Transitions and Critical Phenomena

Memory exponents govern scaling and universality in systems near criticality:

• In Ising-type models with power-law temporal interactions $(t-t')^{-\theta}$, the naive inclusion of memory violates traditional hyperscaling relations. The correct theory introduces a modified effective dimension $d_{\rm eff}=d-1+1/\theta$ and predicts altered mean-field exponents, with the memory exponent $\theta$ controlling dynamical exponent $z=2/\theta$ and other critical indices (Zeng et al., 2022).
• The phase diagram in the $(\theta,d)$-plane reveals a memory-dominated regime where new universality classes and unconventional scaling laws appear. There is a critical threshold in $\theta$ that separates classical (memory-irrelevant), nonclassical, and memory-dominated regimes (Zeng et al., 2022).
• Numerical simulations in 2D and 3D Ising models confirm that for sub-unity $\theta$ critical exponents deviate from their Landau-Ginzburg values.

4. Critical Memory Exponents in Wave Equations and Nonlinear PDEs

In the context of damped wave equations with nonlinear memory

$$u_{tt} + u_t - \Delta u = \int_0^t (t-s)^{-\gamma} |u(s)|^p ds,$$

a critical memory exponent $p_c(n,\gamma)=1+\frac{2(2-\gamma)}{n-2+2\gamma}$ delineates the dividing line between global existence and finite-time blow-up of solutions for given space dimension $n$ and memory kernel exponent $\gamma$ (Fino, 2010):

• For $p>p_c(n,\gamma)$, small data lead to global solutions.
• For $p\leq p_c(n,\gamma)$, solutions blow up in finite time.

As $\gamma\to1$, the kernel becomes short-range, and $p_c\to$ the Fujita critical exponent; as $\gamma\to0$, the kernel is fully long-range, and $p_c\to1+4/n$. Hence $\gamma$ provides a continuous interpolation between instantaneous and strongly nonlocal dynamics, with $p_c$ encoding the “critical memory” threshold (Fino, 2010).

5. Critical Memory Exponents in High-Capacity Associative Memory

In exponential Hopfield networks with storage law $P=\exp(\theta N)$, the memory exponent $\theta$ quantifies the exponential (“superlinear”) scaling of pattern capacity with network size $N$. There exists a sharp critical threshold $\theta_c$ (Albanese et al., 8 Sep 2025):

• For $\theta<\theta_c \approx 0.674$, patterns are stable fixed points and can be reliably retrieved under dynamics.
• For $\theta>\theta_c$, stability is lost and retrieval fails entirely.

$\theta_c$ is determined by a signal-to-noise analysis and corresponds to a mathematical transition in system behavior, analogous to a phase transition. Classical linear Hopfield models only support $P=\alpha N$ with $\alpha\leq\alpha_c\approx 0.138$. The exponential Hopfield family thus reveals fundamentally new scaling governed by the critical memory exponent (Albanese et al., 8 Sep 2025).

6. Memory Exponents in Stochastic Optimization

First-order optimization algorithms with explicit memory, such as memory-$M$ SGD or Heavy Ball variants, exhibit convergence rates tightly controlled by a memory-related schedule exponent $\bar{\alpha}$. For loss decay $L_t\sim t^{-\xi}$ on spectra with power-law eigenvalues, the achievable acceleration is bounded above by a critical memory exponent $\bar{\alpha}_{\text{crit}}=1-1/\nu$, where $\nu$ governs the decay of the smallest eigenvalues (Yarotsky et al., 2024):

• Schedules with $\bar{\alpha}<\bar{\alpha}_{\text{crit}}$ deliver convergence rates $\xi_{\text{new}}=\zeta(1+\bar{\alpha})$ faster than plain GD.
• Exceeding $\bar{\alpha}_{\text{crit}}$ leads to instability or divergence.

Thus, $\bar{\alpha}_{\text{crit}}$ identifies the sharp memory-induced crossover between achievable acceleration and catastrophic instability in stochastic optimization (Yarotsky et al., 2024).

7. Universality and Physical Consequences

The presence and precise value of a critical memory exponent determine:

• Nature of relaxation: Exponential (short memory) vs. power-law (long memory) decay.
• Spectral properties: Emergence of slow tails and excess wings in frequency space.
• Anomalous transport: Sub/super-diffusive transport laws, quantified by the memory exponent.
• Scaling and universality: Breakdown and repair of hyperscaling, creation of new universality classes, and shifting of upper critical dimensions in field theories.
• Capacity and phase transitions: Sharp separatrix between functionally distinct phases (e.g., retrieval ability in neural networks, global existence in PDEs, algorithmic convergence).
• Long-range order and information propagation: In memcomputing and other dynamical systems, critical branching with universal avalanche size exponent $\tau=3/2$ is manifest, and memory exponents regulate self-tuned criticality and efficient search via scale-free cascades (Bearden et al., 2019).

The critical memory exponent is thus a fundamental index controlling system-level qualitative behavior in the presence of memory, connecting mathematical structure, physical phenomena, and computational capacity across disciplines.