Physics-Informed Neural SSMs

• Physics-Informed Neural State-Space Models are deep learning frameworks that integrate first-principles physics (via ODEs, DAEs, or PDEs) directly into neural architectures for dynamic system estimation.
• They combine diverse neural architectures with composite loss functions that balance data fit and physics residuals, using techniques like automatic differentiation and evolutionary search.
• PIN-SSMs achieve robust extrapolation and improved data efficiency across applications in epidemiology, robotics, process engineering, and digital twins while addressing challenges like simplicity bias and uncertainty quantification.

Physics-Informed Neural State-Space Models (PIN-SSMs) are a class of deep learning frameworks that enforce first-principles physics in neural architectures for dynamic systems estimation and prediction. By combining state-space formulations with neural function approximators and embedding physics constraints—typically derived from ODEs, DAEs, or PDEs—directly into the training objective, PIN-SSMs provide interpretable, data-efficient, and robust surrogates applicable to diverse scientific domains. State-space models articulate the evolution of latent system variables under input/control and measurement mappings, permitting hybridization of exact physics and learned components. Modern approaches span feed-forward PINNs for ODEs, PINNODEs leveraging Lagrangian mechanics, transport-PDE-constrained models for spatiotemporal fields, and sub-sequential SSMs mitigating neural simplicity bias in PINN training. This article surveys the architecture, training protocols, underlying dynamical assumptions, and empirical findings defining the PIN-SSM paradigm.

1. Mathematical Formulation and Physical Embedding

PIN-SSMs formalize system dynamics in state-space structure, integrating neural networks as universal approximators and enforcing physics via automatic differentiation. For finite-dimensional ODE systems, the canonical formulation in first-order state-space form is:

$$\dot{z}_1 = z_2,\qquad \dot{z}_2 = -\frac{c}{m}\,z_2 \;-\;\frac{k}{m}\,(z_1 - x_0)$$

as for the damped harmonic oscillator (Linka et al., 2022). The neural surrogate $\hat{x}(t; \theta)$ is parameterized by a multi-layer perceptron and learned on a loss that combines data fit and physics residual,

$$L(\theta, \vartheta) = (1-\varepsilon)L_{\rm data} + \varepsilon L_{\rm phys}$$

where $L_{\rm data}$ is the MSE to measurements and $L_{\rm phys}$ quantifies the compliance of neural outputs with the ODE residual.

For mechanical systems with Euler-Lagrange structure, PINNODE (Roehrl et al., 2020) constructs the total dynamics as: $$M(q)\ddot{q} + C(q, \dot{q})\dot{q} + G(q) = Q^{nc}(q, \dot{q}, u)$$ with $Q^{nc}$ learned via an MLP while $M, C, G$ are fixed by physics. Neural estimation is tightly regularized to satisfy energy identity and non-conservative terms are learned only where analytic models are insufficient.

For high-dimensional transport PDEs, the PIN-SSM paradigm constrains multi-output NNs against local conservation laws for mass, momentum, and energy, via spatial and temporal derivatives: $$\mathcal{R}_{\mathrm{mass}} = \frac{\partial \rho}{\partial t} + \frac{\partial (\rho u)}{\partial z} = 0$$ and analogously for the other conserved quantities (Dave et al., 2023).

2. Network Architectures and Sequence Modeling

PIN-SSMs employ diverse neural architectures aligned to the physical domain and signal topology:

• MLPs with automatic physics differentiation: For scalar ODE surrogates (e.g., COVID-19 outbreak), 2-layer tanh NNs with 32 units/layer are typical (Linka et al., 2022).
• Block-oriented SSMs: Block Hammerstein, Wiener-Hammerstein, and general nonlinear compositions permit separation and interaction of state, input, output, and estimator blocks, with genome-based parameterization of linear/nonlinear features, structured maps (e.g., Soft-SVD, Perron-Frobenius), and activation functions (Skomski et al., 2020).
• Hybrid physics-informed PINNODE: Models non-conservative effects in mechanical systems via 2x50 MLPs, while core dynamics are dictated by exact Lagrangian structure (Roehrl et al., 2020).
• Encoder–decoder multitask networks for transport phenomena: A shared MLP head encodes spatiotemporal location, initial conditions, and controls; decoders produce pressure, velocity, and temperature profiles, each constrained independently via physics residual loss (Dave et al., 2023).
• Sub-sequential state-space neural networks: PINNMamba adapts sequence modeling over moderate-length sub-sequences, utilizing input-dependent (Selective) SSM layers (Mamba-class) to explicitly propagate initial condition information within neural sequence evolution (Xu et al., 1 Feb 2025). This leverages contrastive alignment losses and HiPPO-initialized matrices to overcome simplicity bias.

3. Training Algorithms and Loss Construction

PIN-SSM training integrates physical and observational evidence using composite loss functions:

• Combined data/physics loss: Classical PINNs balance $L_{\rm data}$ and $L_{\rm phys}$ with user-chosen or adaptively learned weight $\varepsilon(t)$, enabling self-adaptive emphasis on physics in data-scarce regimes (Linka et al., 2022).
• Genome-encoded evolutionary search: Asynchronous genetic algorithms sample model design space with mutation/crossover on block structure, linear map, activations, and loss weight genes, optimizing for minimal open-loop MSE on validation populations (Skomski et al., 2020).
• ODE-constrained trajectory loss: PINNODE employs RK4-integrated physics for forward rollout and computes MSE over both position and velocity trajectories, with angular quantities evaluated in polar embedding (Roehrl et al., 2020).
• Physics-informed multitask loss: For PDE fields, total loss includes measurement MSE plus mass, momentum, and energy residual terms, typically log-cosh loss for robustness to sensor noise; loss weights are set to balance physical and empirical fidelity (Dave et al., 2023).
• Sub-sequence PINN loss with contrastive alignment: PINNMamba penalizes residuals on each sequence step and enforces agreement between overlapping sub-sequence predictions, eliminating simplicity bias even for stiff or transport-dominated PDEs (Xu et al., 1 Feb 2025).

4. Empirical Performance and Data Efficiency

PIN-SSMs demonstrate substantial improvements over pure data-driven models in interpolation, extrapolation, and data efficiency:

Model	Interpolation	Extrapolation	Uncertainty Bands	Data Size Efficiency	Physics Fidelity
Data NN	Excellent	Poor	No	Low	Weak
Pure ODE + Bayesian	Good	Good	Yes	High	Strong
PINN	Good	Good	No	High	Strong
Self-Adaptive PINN	Robust	Good	No	Robust	Strong
Bayesian PINN	Good	Good	Yes	Robust	Strong
PINNODE	High	Plausible	N/A	Higher	Guaranteed
PSM (Transport)	High	High	N/A	Higher	Guaranteed
PINNMamba	Superior	Superior	N/A	Superior	Bypasses bias

Quantitatively, reductions in RMSE versus non-physics NNs can be as high as 94% in temperature field prediction (Dave et al., 2023), 86% on canonical PDE benchmarks (Xu et al., 1 Feb 2025), and over 40× in open-loop error for mechanical systems (Roehrl et al., 2020). PINNs also yield order-of-magnitude speedups in power transient stability (100×‒1000× over RK45), due to direct neural evaluation instead of time-stepping (Stiasny et al., 2021).

5. Uncertainty Quantification via Bayesian PIN-SSMs

Bayesian extensions of PINNs and PIN-SSMs place probabilistic priors on physics parameters and neural weights. Posterior uncertainty is drawn via Hamiltonian Monte Carlo (e.g., NUTS, PyMC3/TensorFlow Probability). The joint posterior is

$$\pi(\theta, \vartheta) \propto p(\{x_{\rm data}\}, \{r\} | \theta, \vartheta) \pi(\theta) \pi(\vartheta)$$

with likelihoods on both data fit and physics residual, enabling credible intervals on trajectories and physics parameters. Bayesian PINNs maintain narrow credible bands even outside the data region, whereas Bayesian NNs without physics constraints produce wide, uninformative intervals (Linka et al., 2022).

6. Practical Applications and Digital Twins

PIN-SSMs underpin advances in several domains:

• Epidemiology: Bayesian PINNs for COVID-19 outbreak enable interpolation and physically plausible forecasting over incomplete, noisy datasets, and data-driven inference of epidemic parameters (Linka et al., 2022).
• Robotics & Control: PINNODE achieves physically consistent cart-pole dynamics modeling with robust prediction under sensor noise, outperforming pure ODE fitting or black-box NNs in trajectory tracking and stability (Roehrl et al., 2020).
• Process Engineering: PSM architectures deliver sub-1% relative error in real-time control and diagnostics of heated channels and cooling loops, supporting supervisory command governors and fault diagnosis using physics residual fields (Dave et al., 2023).
• Power Systems: PINN surrogates facilitate transient stability assessment for large-scale grid models, accelerating “what-if” screening and critical contingency detection by orders of magnitude (Stiasny et al., 2021).
• Scientific Computing: Sub-sequence PINN frameworks for PDE solvers generalize well to high-dimensional transport, convection, reaction, and wave equations, mitigating over-smoothness and propagation failures present in vanilla PINNs (Xu et al., 1 Feb 2025).

Digital Twin implementations evolve PIN-SSMs online, constantly updating model weights $\theta$ and PDE coefficients based on streaming sensor data. This supports tracking equipment degradation and adaptive control in real-world processes.

7. Limitations, Failure Modes, and Research Directions

PIN-SSMs are subject to several challenges:

• Simplicity bias and non-propagation of initial conditions: Vanilla PINNs can relax to low-frequency smooth solutions, failing on transport or hyperbolic equations. The SSM and sub-sequence alignment strategy of PINNMamba effectively eliminate these failure modes (Xu et al., 1 Feb 2025).
• Continuous–discrete mismatch: Discrete enforcement of physical residuals can permit globally incorrect solutions that satisfy sampled constraints. Ensuring physical propagation across the domain requires careful model and loss design.
• Loss weighting and heuristic tuning: Adaptive schemes for data–physics loss balance can destabilize training. Self-adaptive PINNs and gradient-based auto-balancing partially mitigate this but formal guarantees are elusive (Linka et al., 2022, Xu et al., 1 Feb 2025).
• High-dimensional inputs: Applications to large systems (e.g., power networks, 2D/3D PDEs) can strain neural representations and GPU memory, motivating research in low-rank encodings, sliding windows, and distributed sequence models (Xu et al., 1 Feb 2025).
• Uncertainty quantification scalability: Bayesian PINNs incur significant computational expense and require careful scaling of physics contributions to maintain tractable posterior sampling (Linka et al., 2022).
• Generalization theory: Zero physics residual at collocation points does not imply globally correct solutions. Open problems include formal convergence guarantees and robust out-of-sample behavior (Stiasny et al., 2021).

Ongoing research focuses on architecture search via genome encoding, adaptive sequence length modeling, Bayesian SSMs for uncertainty quantification, and extensions to irregular domains and nonuniform discretization.

---

References:

• (Linka et al., 2022) Bayesian Physics-Informed Neural Networks for real-world nonlinear dynamical systems
• (Roehrl et al., 2020) Modeling System Dynamics with Physics-Informed Neural Networks Based on Lagrangian Mechanics
• (Dave et al., 2023) Physics-informed State-space Neural Networks for Transport Phenomena
• (Skomski et al., 2020) Physics-Informed Neural State Space Models via Learning and Evolution
• (Stiasny et al., 2021) Transient Stability Analysis with Physics-Informed Neural Networks
• (Xu et al., 1 Feb 2025) Sub-Sequential Physics-Informed Learning with State Space Model