A Step Towards Uncovering The Structure of Multistable Neural Networks

Abstract: We study how the connectivity within a recurrent neural network determines and is determined by the multistable solutions of network activity. To gain analytic tractability we let neural activation be a non-smooth Heaviside step function. This nonlinearity partitions the phase space into regions with different, yet linear dynamics. In each region either a stable equilibrium state exists, or network activity flows to outside of the region. The stable states are identified by their semipositivity constraints on the synaptic weight matrix. The restrictions can be separated by their effects on the signs or the strengths of the connections. Exact results on network topology, sign stability, weight matrix factorization, pattern completion and pattern coupling are derived and proven. Our work may lay the foundation for multistability in more complex recurrent neural networks.