Variational approach to unsupervised learning

Abstract: Deep belief networks are used extensively for unsupervised stochastic learning on large datasets. Compared to other deep learning approaches their layer-by-layer learning makes them highly scalable. Unfortunately, the principles by which they achieve efficient learning are not well understood. Numerous attempts have been made to explain their efficiency and applicability to a wide class of learning problems in terms of principles drawn from cognitive psychology, statistics, information theory, and more recently physics, but quite often these imported principles lack strong scientific foundation. Here we demonstrate how one can arrive at convolutional deep belief networks as potential solution to unsupervised learning problems without making assumptions about the underlying framework. To do this, we exploit the notion of symmetry that is fundamental in machine learning, physics and other fields, utilizing the particular form of the functional renormalization group in physics.