Structure Learning of Markov Random Fields through Grow-Shrink Maximum Pseudolikelihood Estimation

Abstract: Learning the structure of Markov random fields (MRFs) plays an important role in multivariate analysis. The importance has been increasing with the recent rise of statistical relational models since the MRF serves as a building block of these models such as Markov logic networks. There are two fundamental ways to learn structures of MRFs: methods based on parameter learning and those based on independence test. The former methods more or less assume certain forms of distribution, so they potentially perform poorly when the assumption is not satisfied. The latter can learn an MRF structure without a strong distributional assumption, but sometimes it is unclear what objective function is maximized/minimized in these methods. In this paper, we follow the latter, but we explicitly define the optimization problem of MRF structure learning as maximum pseudolikelihood estimation (MPLE) with respect to the edge set. As a result, the proposed solution successfully deals with the {\em symmetricity} in MRFs, whereas such symmetricity is not taken into account in most existing independence test techniques. The proposed method achieved higher accuracy than previous methods when there were asymmetric dependencies in our experiments.