Structure-Preserving Neural Ordinary Differential Equations for Stiff Systems

Abstract: Neural ordinary differential equations (NODEs) are an effective approach for data-driven modeling of dynamical systems arising from simulations and experiments. One of the major shortcomings of NODEs, especially when coupled with explicit integrators, is its long-term stability, which impedes their efficiency and robustness when encountering stiff problems. In this work we present a structure-preserving NODE approach that learns a transformation into a system with a linear and nonlinear split. It is then integrated using an exponential integrator, which is an explicit integrator with stability properties comparable to implicit methods. We demonstrate that our model has advantages in both learning and deployment over standard explicit or even implicit NODE methods. The long-time stability is further enhanced by the Hurwitz matrix decomposition that constrains the spectrum of the linear operator, therefore stabilizing the linearized dynamics. When combined with a Lipschitz-controlled neural network treatment for the nonlinear operator, we show the nonlinear dynamics of the NODE are provably stable near a fixed point in the sense of Lyapunov. For high-dimensional data, we further rely on an autoencoder performing dimensionality reduction and Higham's algorithm for the matrix-free application of the matrix exponential on a vector. We demonstrate the effectiveness of the proposed NODE approach in various examples, including the Robertson chemical reaction problem and the Kuramoto-Sivashinky equation.