Non-Euclidean Contraction Analysis of Continuous-Time Neural Networks

Abstract: Critical questions in dynamical neuroscience and machine learning are related to the study of continuous-time neural networks and their stability, robustness, and computational efficiency. These properties can be simultaneously established via a contraction analysis. This paper develops a comprehensive non-Euclidean contraction theory for continuous-time neural networks. Specifically, we provide novel sufficient conditions for the contractivity of general classes of continuous-time neural networks including Hopfield, firing rate, Persidskii, Lur'e, and other neural networks with respect to the non-Euclidean $\ell_1/\ell_\infty$ norms. These sufficient conditions are based upon linear programming or, in some special cases, establishing the Hurwitzness of a particular Metzler matrix. To prove these sufficient conditions, we develop novel results on non-Euclidean logarithmic norms and a novel necessary and sufficient condition for contractivity of systems with locally Lipschitz dynamics. For each model, we apply our theoretical results to compute the optimal contraction rate and corresponding weighted non-Euclidean norm with respect to which the neural network is contracting.