Network Horizon Dynamics I: Qualitative Aspects

Abstract: Mostly acyclic directed networks, treated mathematically as directed graphs, arise in machine learning, biology, social science, physics, and other applications. Newman [1] has noted the mathematical challenges of such networks. In this series of papers, we study their connectivity properties, focusing on three types of phase transitions that affect horizon sizes for typical nodes. The first two types involve the familiar emergence of giant components as average local connectivity increases, while the third type involves small-world horizon growth at variable distance from a typical node. In this first paper, we focus on qualitative behavior, simulations, and applications, leaving formal considerations for subsequent papers. We explain how such phase transitions distinguish deep neural networks from shallow machine learning architectures, and propose hybrid local/random network designs with surprising connectivity advantages. We also propose a small-world approach to the horizon problem in the cosmology of the early universe as a novel alternative to the inflationary hypothesis of Guth and Linde.