Covariance spectrum in nonlinear recurrent neural networks

Abstract: Advances in simultaneous recordings of large numbers of neurons have driven significant interest in the structure of neural population activity such as dimension. A key question is how these dynamic features arise mechanistically and their relationship to circuit connectivity. It was previously proposed to use the covariance eigenvalue distribution, or spectrum, which can be analytically derived in random recurrent networks, as a robust measure to describe the shape of neural population activity beyond the dimension (Hu and Sompolinsky 2022). Applications of the theoretical spectrum have broadly found accurate matches to experimental data across brain areas providing mechanistic insights into the observed low dimensional population dynamics (Morales et al. 2023). However, the empirical success highlights a gap in theory, as the neural network model used to derive the spectrum was minimal with linear neurons. In this work, we aim to close this gap by studying the covariance spectrum in networks with nonlinear neurons and under broader dynamical regimes including chaos. Surprisingly, we found that the spectrum can be precisely understood by equations analogous to the linear theory substituted with an effective recurrent connection strength parameter, that reflects both the connection weights and the nonlinearity of neurons. Across dynamical regimes, this effective connection strength provides a unified interpretation for the spectrum and dimension changes, and stays near the critical value in the chaotic regime without fine-tuning. These results further our understanding of nonlinear neural population dynamics and provide additional theoretical support for applying the covariance spectrum analysis in biological circuits.