Stochastic Thermodynamics of Learning

Abstract: Virtually every organism gathers information about its noisy environment and builds models from that data, mostly using neural networks. Here, we use stochastic thermodynamics to analyse the learning of a classification rule by a neural network. We show that the information acquired by the network is bounded by the thermodynamic cost of learning and introduce a learning efficiency $\eta\le1$. We discuss the conditions for optimal learning and analyse Hebbian learning in the thermodynamic limit.