Discrete signaling mediates chaotic regularization in recurrent neural networks

Abstract: Cortical circuits operate in a regime of intrinsic chaos, where even tiny changes in input can lead to divergent neural responses. Yet, remarkably, population codes in the brain vary smoothly with sensory stimuli, forming coherent representational manifolds. How can chaotic networks sustain such stable coding? Here, we develop a theoretical framework that links the microscopic chaos of recurrent networks to the macroscopic geometry of neural representations. Combining kernel methods with dynamical mean-field theory, we show that chaotic dynamics induce local roughness (introducing sharp distortions at small scales) while preserving global smoothness across larger stimulus variations. This structural property acts as an intrinsic regularizer, enhancing generalization while maintaining expressivity. Moreover, we show how chaotic networks naturally produce power-law spectral signatures, closely matching experimental observations in cortical recordings. These results explain how chaotic spiking networks can sustain smooth, differentiable population codes and establish a theoretical framework linking network dynamics, computational structure, and recorded neural activity.

• The paper shows that discrete signaling induces abrupt local changes ('local roughness') while maintaining smooth global population codes in RNNs.
• It introduces a two-replica DMFT framework to derive time-dependent neural kernel functions that connect microscopic chaos with macroscopic regularization.
• Computational experiments illustrate that chaotic discrete networks improve generalization by suppressing small-scale overfitting, aligning with observed neurophysiological spectra.

Discrete Signaling Mediates Chaotic Regularization in Recurrent Neural Networks

Overview and Theoretical Framework

The paper "Discrete signaling mediates chaotic regularization in recurrent neural networks" (2606.04426) investigates the paradox of how cortical circuits, which exhibit intrinsic chaos at the microscopic (single-neuron) level, manage to sustain smooth and reliable neural population codes at the macroscopic scale. Despite small input fluctuations causing large divergences—reflecting positive Lyapunov exponents and deterministic chaos—experimental recordings have revealed that sensory representations in the cortex are globally smooth and organized along low-dimensional manifolds. This study constructs a precise theoretical bridge connecting microscopic chaotic dynamics in recurrent networks to the geometry and statistics of population codes. The authors leverage and extend kernel methods and dynamical mean-field theory (DMFT), introducing a two-replica DMFT formalism capable of capturing the local and global properties of neural codes implemented by chaotic dynamics.

The analysis provides a unified perspective on both continuous-rate and discrete-state (i.e., binary or spiking-like) recurrent network models, quantifying how discrete signaling induces distinct regularization effects. The theoretical results are expressed in terms of the time-evolved neural kernel, which underlies both Bayesian inference and ridge regression in the reservoir computing paradigm. The work further provides a direct quantitative connection between neural chaos, the architecture and balance of input versus recurrent drive, and empirically observed eigenspectra in population activity recordings.

Discrete Versus Continuous Signaling: Model Construction and Kernel Regimes

The study systematically contrasts two archetypal models of recurrent signaling:

• Discrete (binary) networks: Units switch asynchronously between two states via Glauber dynamics, closely modeling spike-like all-or-nothing events.
• Continuous (rate) networks: Units follow noisy continuous-valued ODEs, approximating population-averaged rate codes.

The paper rigorously constructs the mean-field dynamics for both regimes and highlights that, although matched in first and second statistical moments for a single input (autocorrelation), they diverge strongly in their cross-trajectory (replica) correlations. This divergence is crucial, as it governs the transformation of stimulus similarity into output similarity—encapsulated by the neural kernel.

A key technical contribution is the explicit derivation of time-dependent kernel functions for both models using two-replica DMFT. For discrete networks, the cross-trajectory correlations exhibit sharp non-analyticities, manifesting as a discontinuity (jump) at vanishing input difference. This is absent in continuous models unless external noise is strong.

Chaos-Induced Regularization: Local Roughness and Global Coherence

The researchers analyze how chaotic dynamics affect the geometric properties of neural codes, focusing on implications for learning and generalization:

• Local effects: In discrete networks, infinitesimal input changes lead to abrupt network state changes—a phenomenon the authors term “local roughness.” Formally, the kernel function drops discontinuously at smallest input separations, corresponding to infinite local Lyapunov exponents in the large-$N$ limit.
• Global effects: Despite local instability, input similarities on larger scales are preserved—global structure of the input manifold is not fragmented. Thus, the chaotic dynamics act as an intrinsic regularizer: while preserving expressivity and large-scale smoothness, small-scale overfitting and memorization are suppressed.

The kernel jump acts as an implicit label noise prior of variance $\Delta$, echoing the benign overfitting regime known from linear models. Numerical experiments on tasks such as image classification (e.g., CIFAR10) demonstrate that chaotic discrete networks attain high accuracy despite rough, non-differentiable internal codes.

Spectral Properties and Empirical Implications

A central focus is on how network chaos and signaling type shape the eigenspectrum of neural covariance and kernel matrices:

• Spectral analysis: The mean-field kernel admits a Mercer decomposition, with eigenvalues $\lambda_n$ governing the sample-wise prioritization of smooth versus rough functions.
• Discrete networks: The non-analytic kernel, steep at the coincidence point, yields a flat, power-law eigenspectrum, boosting high-frequency components. As observed in neurophysiological data, eigenvalue spectra decay as $\lambda_n \sim n^{-\nu}$, with $\nu$ close to unity—a signature linked here directly to chaotic regularization.
• Continuous networks: By tuning synaptic strength $g$, one can transition from a regular regime (single dominant eigenmode) to a moderately chaotic regime (rich multi-scale spectrum), and further to a strongly chaotic regime where expressivity is lost as all correlations are quickly destroyed.

The theoretical framework predicts that the slope $\nu$ of the neural covariance spectrum is tunable via the ratio of recurrent to input projection strength, and that recurrently driven regimes yield spectra closely matching those observed in high-dimensional neural recordings [Stringer et al.].

Computational Implications and Future Directions

By formalizing the connection between chaos, kernel geometry, and generalization, the work advances the theoretical underpinnings of how biological and artificial recurrent neural networks balance stability, expressivity, and robustness:

• Chaos as regularization: Strong discrete chaos prevents memorization of microscopic fluctuations, leading to robust extrapolation whilst maintaining global interpolative capabilities. This resolves prior contradictions about the negative impact of chaos on computational reliability.
• Memory and multi-scale computing: Moderately chaotic continuous-rate networks support persistent input-dependent activity and enable multi-scale computation, essential for temporal sequence processing and reservoir computing paradigms.
• Link to brain data: The explanation for observed power-law population spectra in cortex is placed on solid theoretical footing, connecting microscopic synaptic architecture to macroscopic, computationally relevant geometry.
• Limitations and extensions: The current approach addresses only regimes with fixed random connectivity (lazy training) and static inputs. Extending the analysis to plastic recurrent weights and temporal sequence learning remains an open challenge, as does bridging the binary model to more realistic spiking neurons.

Conclusion

This study rigorously demonstrates that discrete signaling in chaotic recurrent networks leads to an emergent, deterministic form of regularization, reconciling locally rough (non-differentiable) codes with the global smoothness required for reliable population coding. The kernel-based mean-field theory predicts both the computational properties and empirical spectral features of neural codes, supporting a theoretical synthesis of dynamical systems and statistical learning perspectives in neuroscience and machine learning. The work sets the stage for further studies on the interplay between chaos, structure, and learning in high-dimensional recurrent systems and their biological counterparts.