Matrix Lie Maps and Neural Networks for Solving Differential Equations

Abstract: The coincidence between polynomial neural networks and matrix Lie maps is discussed in the article. The matrix form of Lie transform is an approximation of the general solution of the nonlinear system of ordinary differential equations. It can be used for solving systems of differential equations more efficiently than traditional step-by-step numerical methods. Implementation of the Lie map as a polynomial neural network provides a tool for both simulation and data-driven identification of dynamical systems. If the differential equation is provided, training a neural network is unnecessary. The weights of the network can be directly calculated from the equation. On the other hand, for data-driven system learning, the weights can be fitted without any assumptions in view of differential equations. The proposed technique is discussed in the examples of both ordinary and partial differential equations. The building of a polynomial neural network that simulates the Van der Pol oscillator is discussed. For this example, we consider learning the dynamics from a single solution of the system. We also demonstrate the building of the neural network that describes the solution of Burgers' equation that is a fundamental partial differential equation.

• The paper introduces a novel framework using matrix Lie maps and polynomial neural networks to bypass traditional step-by-step numerical integration for ODEs.
• It details a methodology where neural network weights are computed directly from differential equations or learned from data, ensuring precise system evolution.
• Applications to the Van der Pol oscillator and Burgers' equation demonstrate lower mean squared error and improved computational efficiency compared to classic methods.

Solving Differential Equations with Matrix Lie Maps and Neural Networks

This paper introduces a novel approach for solving differential equations by combining polynomial neural networks with matrix Lie maps. The core idea involves representing the solution of nonlinear ordinary differential equations (ODEs) using a matrix form of the Lie transform, which can then be implemented as a polynomial neural network. This method offers advantages over traditional step-by-step numerical methods, particularly for systems where computational efficiency is crucial.

Methodology

The approach hinges on the representation of dynamical systems via nonlinear ODEs:

$$\frac{d}{dt}X = F(t, X) = \sum_{k=0}^{\infty} P^{1k}(t)X^{[k]},$$

where $X \in R^n$ is the state vector, and $X^{[k]}$ denotes the k-th Kronecker power of $X$. The solution to this system can be expressed as a series:

$$X(t|t_0) = \mathcal{M}(t|t_0) \circ X_0 = \sum_{k=0}^{\infty} M^{1k}(t|t_0)X_0^{[k]},$$

where $X_0 = X(t_0)$. The matrices $M^{1k}$ are calculated using the equation:

$$\frac{d}{dt} M^{ik}(t|t_0) = \sum_{j=i}^{k} P^{ij}(t)M^{jk}(t|t_0), \; 1 \leq i < k.$$

This transformation, $\mathcal{M}$, approximates the evolution operator of the system, allowing the state vector's evolution to be calculated directly, bypassing step-by-step integration. The neural network implements the map $\mathcal{M}: X \rightarrow Y$ in the form:

$Y = W_0 + W_1X + W_2X^{[2]} + \ldots + W_kX^{[k]},$

where $W_i$ are weight matrices corresponding to $M^{1k}$.

Figure 1: Neural network representation of third order matrix Lie map.

Implementation and Results

The authors demonstrate the effectiveness of this approach through two key examples: the Van der Pol oscillator and Burgers' equation. For the Van der Pol oscillator, the neural network's weights are computed directly from the equation, and the simulation results closely match those obtained using traditional numerical methods like the implicit Adams method (Figure 2).

Figure 2: Simulation of the Van der Pol oscillator. Red lines for the implicit Adams method of eighth order, blue dots for matrix Lie map of third order.

Furthermore, the paper explores a data-driven approach where the weights of the neural network are fitted using a time series of the system's evolution. This allows the network to learn the dynamics of the system without explicit knowledge of the underlying equations (Figure 3).

Figure 3: Training data for the neural network (red dots) and provided predictions (lines).

The results show that the network can accurately predict the system's behavior for different initial conditions.

The paper also addresses Burgers' equation, a fundamental PDE, by converting it into a system of ODEs and applying the matrix Lie map approach. The Lie transform-based neural network demonstrates better accuracy and computational performance compared to traditional finite difference methods (FDM) (Figure 4). Specifically, the neural network achieves a lower mean squared error (MSE) in less computational time, highlighting its efficiency.

Figure 4: A benchmark (a) on mesh $1000\times 2000$ and Lie transform--based neural network (b) on mesh $1000\times500$.

Figure 5: Numerical schemes for FDM and Lie transform--based neural network for (\ref{burgers_de}).

Implications and Future Directions

This research offers a promising alternative to traditional numerical methods for solving differential equations, particularly in scenarios demanding high computational performance. The ability to directly calculate the neural network's weights from the equations, or to learn them from data, provides flexibility in modeling various dynamical systems. The successful application to both ODEs and PDEs underscores the method's versatility.

The paper identifies several areas for future research, including the limitations of the data-driven approach for large-scale systems, optimal network configuration, and the handling of noisy data. Further exploration in these directions could significantly expand the applicability and robustness of this technique.

Conclusion

The integration of matrix Lie maps with polynomial neural networks presents a compelling framework for solving differential equations. This method not only offers computational advantages but also opens new avenues for data-driven system identification and simulation. The results presented in the paper highlight the potential of this approach for a wide range of applications in science and engineering.