Activity–degree scaling as the cause of poor size generalization in nODEs

Prove that, for neural ordinary differential equations with Barabási–Barzel–form vector fields trained on small graphs, an increase of node-state magnitude with node degree in the underlying dynamical system causes predictions at high-degree nodes on larger, degree-heterogeneous graphs to enter regions of state space not covered by the training data and thereby degrades predictive performance.

Background

The paper shows empirically that neural ODEs tailored to Barabási–Barzel dynamics generalize well across size on degree-homogeneous graphs but often fail on degree-heterogeneous graphs, except for systems like SIS and MAK. In Appendix A, the authors derive that in Michaelis–Menten, neuronal, and birth–death models the magnitude of a node’s state grows with its degree, while it remains bounded or decreases in SIS and MAK.

Based on these observations, the discussion hypothesizes a mechanistic explanation: when deploying a model trained on small graphs to larger, degree-heterogeneous graphs, high-degree hubs can reach state magnitudes outside the training domain, leading to out-of-distribution predictions and increased error. Formalizing and proving this mechanism would explain the observed dependence of size generalization on graph degree heterogeneity and the type of dynamics.