Physics-informed Neural Networks for Functional Differential Equations: Cylindrical Approximation and Its Convergence Guarantees

Abstract: We propose the first learning scheme for functional differential equations (FDEs). FDEs play a fundamental role in physics, mathematics, and optimal control. However, the numerical analysis of FDEs has faced challenges due to its unrealistic computational costs and has been a long standing problem over decades. Thus, numerical approximations of FDEs have been developed, but they often oversimplify the solutions. To tackle these two issues, we propose a hybrid approach combining physics-informed neural networks (PINNs) with the \textit{cylindrical approximation}. The cylindrical approximation expands functions and functional derivatives with an orthonormal basis and transforms FDEs into high-dimensional PDEs. To validate the reliability of the cylindrical approximation for FDE applications, we prove the convergence theorems of approximated functional derivatives and solutions. Then, the derived high-dimensional PDEs are numerically solved with PINNs. Through the capabilities of PINNs, our approach can handle a broader class of functional derivatives more efficiently than conventional discretization-based methods, improving the scalability of the cylindrical approximation. As a proof of concept, we conduct experiments on two FDEs and demonstrate that our model can successfully achieve typical $L^1$ relative error orders of PINNs $\sim 10^{-3}$. Overall, our work provides a strong backbone for physicists, mathematicians, and machine learning experts to analyze previously challenging FDEs, thereby democratizing their numerical analysis, which has received limited attention. Code is available at \url{https://github.com/TaikiMiyagawa/FunctionalPINN}.

• The paper introduces a hybrid PINN and cylindrical approximation method that transforms complex FDEs into manageable PDEs with proven convergence.
• It achieves high accuracy with L1 relative errors around 10⁻³ on benchmarks like the functional transport and Burgers-Hopf equations.
• The method significantly reduces computational complexity, broadening access to advanced FDE analysis in scientific and engineering applications.

Physics-Informed Neural Networks for Functional Differential Equations: Cylindrical Approximation and Its Convergence Guarantees

The paper under review introduces a pioneering approach to addressing the longstanding computational challenges associated with Functional Differential Equations (FDEs). FDEs are essential in numerous scientific and engineering domains, spanning physics, mathematics, and optimal control. However, their numerical analysis has traditionally been hampered by excessive computational costs. The authors propose a novel hybrid methodology that merges Physics-Informed Neural Networks (PINNs) with cylindrical approximation, aiming to both efficiently handle and extend the class of solvable FDEs.

Methodology Overview

The authors' primary contribution lies in employing a cylindrical approximation, which involves expanding functions and their derivatives into a high-dimensional Partial Differential Equation (PDE) framework using an orthonormal basis. This expansion translates complex FDEs into more tractable PDEs, which are subsequently tackled using PINNs.

A significant advance presented in the paper is the proof of convergence theorems, ensuring the reliability of the cylindrical approximation in approximating functional derivatives and solutions. This theoretical foundation guarantees that the approximations achieved by this method will converge to the true solutions as the dimension of the approximation increases.

Numerical Results

The efficacy of the proposed approach is demonstrated through experiments on two specific FDEs: the functional transport equation and the Burgers-Hopf equation. The results are promising, achieving $L^1$ relative errors on the order of $10^{-3}$, which is a typical accuracy level for PINN-based solutions. This performance underscores the capability of the method to maintain precision despite the dimensionality and complexity of the problems addressed.

Theoretical and Practical Implications

This work carries substantial implications both theoretically and practically. Theoretically, it offers a rigorous framework for ensuring the convergence of cylindrical approximations, filling a gap in existing literature. Practically, the approach extends the boundaries of tractable FDEs in computational settings, effectively democratizing analysis tools that were previously computationally prohibitive.

Computational Complexity

The approach reduces the computational complexity significantly, typically to $O(m^r)$, where $m$ is the degree of approximation, and $r$ represents the order of the derivatives involved (usually 1 or 2). This is a notable improvement over existing methods, allowing for higher expressivity with increased dimensionality without exponentially increasing computational costs.

Future Directions

Future research could explore extending this methodology to other types of FDEs, particularly those arising in more complex domains such as quantum field theory. Additionally, improving the robustness and accuracy of PINN integration will be crucial in further enhancing the proposed approach's effectiveness.

Overall, this research contributes significantly to the field of functional analysis and computational mathematics, providing both a theoretical foundation and practical tools for advancing the numerical analysis of FDEs. With continued development, this methodology holds the promise of broadening the scope and accessibility of FDE solutions in scientific research and engineering applications.