Interpreting Deep Learning by Establishing a Rigorous Corresponding Relationship with Renormalization Group

Abstract: In this paper, we focus on the interpretability of deep neural network. Our work is motivated by the renormalization group (RG) in statistical mechanics. RG plays the role of a bridge connecting microscopical properties and macroscopic properties, the coarse graining procedure of it is quite similar with the calculation between layers in the forward propagation of the neural network algorithm. From this point of view we establish a rigorous corresponding relationship between the deep neural network (DNN) and RG. Concretely, we consider the most general fully connected network structure and real space RG of one dimensional Ising model. We prove that when the parameters of neural network achieve their optimal value, the limit of coupling constant of the output of neural network equals to the fixed point of the coupling constant in RG of one dimensional Ising model. This conclusion shows that the training process of neural network is equivalent to RG and therefore the network extract macroscopic feature from the input data just like RG.