Exact Enforcement of Temporal Continuity in Sequential Physics-Informed Neural Networks

Abstract: The use of deep learning methods in scientific computing represents a potential paradigm shift in engineering problem solving. One of the most prominent developments is Physics-Informed Neural Networks (PINNs), in which neural networks are trained to satisfy partial differential equations (PDEs). While this method shows promise, the standard version has been shown to struggle in accurately predicting the dynamic behavior of time-dependent problems. To address this challenge, methods have been proposed that decompose the time domain into multiple segments, employing a distinct neural network in each segment and directly incorporating continuity between them in the loss function of the minimization problem. In this work we introduce a method to exactly enforce continuity between successive time segments via a solution ansatz. This hard constrained sequential PINN (HCS-PINN) method is simple to implement and eliminates the need for any loss terms associated with temporal continuity. The method is tested for a number of benchmark problems involving both linear and non-linear PDEs. Examples include various first order time dependent problems in which traditional PINNs struggle, namely advection, Allen-Cahn, and Korteweg-de Vries equations. Furthermore, second and third order time-dependent problems are demonstrated via wave and Jerky dynamics examples, respectively. Notably, the Jerky dynamics problem is chaotic, making the problem especially sensitive to temporal accuracy. The numerical experiments conducted with the proposed method demonstrated superior convergence and accuracy over both traditional PINNs and the soft-constrained counterparts.

• The paper introduces HCS-PINNs which exactly enforce temporal continuity by segmenting the time domain and using interpolation functions to eliminate continuity loss terms.
• The methodology is validated on benchmark PDEs—including advection, wave, Allen-Cahn, KdV, and chaotic systems—yielding errors an order of magnitude lower than traditional approaches.
• By removing manual tuning for temporal constraints, HCS-PINNs reduce computational overhead and simplify training in complex time-dependent problems.

Exact Enforcement of Temporal Continuity in Sequential Physics-Informed Neural Networks

Introduction

The research presented in the paper addresses critical challenges associated with Physics-Informed Neural Networks (PINNs) in solving time-dependent Partial Differential Equations (PDEs). Standard PINNs often face difficulties in predicting the dynamic behavior of time-dependent problems due to their complex loss landscapes and causality violations that arise when treating temporal variables similarly to spatial ones. The proposed methodology, Hard Constrained Sequential Physics-Informed Neural Networks (HCS-PINNs), advances the PINN framework by introducing exact enforcement of continuity between time segments, eliminating the need for temporal continuity loss terms.

Methodology

Temporal Domain Decomposition

The proposed HCS-PINN method divides the time domain into multiple segments, employing distinct neural networks for each segment. Temporal continuity is precisely ensured by integrating a specific solution ansatz, thereby avoiding loss terms traditionally used to approximate this continuity. This approach simplifies the optimization process by focusing solely on minimizing the PDE residuals.

Figure 1: Collocation points for PDE, initial and boundary conditions.

Architectural Overview

The architecture incorporates auxiliary interpolation functions to achieve desired continuity across time segments. These functions ensure $C^0$, $C^1$, or higher-order continuity based on the problem's requirements. The approach is particularly beneficial for problems requiring $C^m$ continuity, where $m$ is dictated by the PDE's highest time derivative.

Figure 2: Interpolation functions for $C^0$, $C^1$, and $C^2$ continuity.

Numerical Experiments and Results

The experimental setup evaluated HCS-PINNs across multiple benchmark PDEs: the advection equation, wave equation, Allen-Cahn, Korteweg-de Vries (KdV) equations, and the chaotic Jerky dynamics equation.

Advection and Wave Equations: These linear, hyperbolic equations exhibit high sensitivity to numerical methods at large characteristic speeds. The introduction of HCS-PINN resulted in relative $L^2$ errors an order of magnitude lower than those produced by SCS-PINNs, particularly under high velocity conditions.

Figure 3: HCS-PINN and SCS-PINN solution of advection equation for nt = 10.

Allen-Cahn and KdV Equations: These non-linear equations serve as standard benchmarks. HCS-PINNs achieved superior accuracy without requiring extensive hyperparameter tuning, showcasing its robustness for complex nonlinear dynamics.

Jerky Dynamics Equation: As a representative of chaotic systems, this third-order PDE in time demonstrated HCS-PINN's capability to maintain solution accuracy over extensive temporal domains where traditional approaches struggle.

Figure 4: Evolution of Jerky dynamics in 3D phase space with HCSPINN (nt=10).

Practical and Theoretical Implications

HCS-PINNs represent a significant improvement over traditional PINNs, particularly in maintaining causality and reducing computational overhead by avoiding manual or heuristic adjustment of temporal continuity constraints. The approach facilitates more robust and accurate solutions for a wide range of temporal PDEs, from linear transport equations to complex non-linear and chaotic systems.

Conclusion

The introduction of HCS-PINN marks a substantial advancement in the deployment of neural networks for solving time-dependent PDEs. By exactly enforcing temporal continuity, the method circumvents the limitations of traditional and soft-constrained PINNs, providing a more resilient and efficient solution framework. Future research should explore extensions to multi-dimensional space-time PDEs and further integration with advanced deep learning architectures to enhance scalability and performance.