Physics-Informed Stochastic Perturbation Scheme

• Physics-Informed Stochastic Perturbation Scheme is a framework that combines stochastic modeling, data-driven inference, and physical laws to solve differential equations.
• It employs neural networks with tailored loss functions and noise injection mechanisms to accurately infer hidden dynamics and governing parameters.
• The approach enhances model generalization and robustness by embedding physical priors and error bounds, making it effective for uncertainty quantification and inverse problems.

A physics-informed stochastic perturbation scheme integrates stochastic modeling, data-driven inference, and physical law constraints to solve direct and inverse stochastic problems governed by differential equations. Such schemes introduce stochasticity by perturbing system states or network outputs in accordance with known physics (e.g., SDEs, Fokker–Planck formalisms, or conservation laws), embedding these constraints in machine learning models—most notably physics-informed neural networks (PINNs). These frameworks enable the simultaneous inference of hidden dynamics, probabilistic states, or governing parameters, leveraging minimal, often sparse, stochastic observations. The approach has achieved prominence for its rigorous mathematical regularity, robust generalization, and capacity to encode domain-specific physical priors.

1. Mathematical Foundation: SDEs and Fokker–Planck Equations

The foundational setting is a stochastic system, typically governed by an SDE of the form

$$dX_t = a(X_t)\,dt + \sigma(X_t)\,dB_t + \varepsilon\,dL^\alpha_t$$

where $a(x)$ is the drift, $\sigma(x)$ the (possibly state-dependent) diffusion, $B_t$ Brownian motion, and $L^\alpha_t$ an $\alpha$-stable Lévy process with intensity $\varepsilon$ (Chen et al., 2020). The evolution of the state probability density $p(x,t)$ is described via the forward Kolmogorov (Fokker–Planck) equation,

$\partial_t\,p(x,t) = \mathcal{A}^*\,p(x,t)$

with

$$\mathcal{A}^*\,p = -\sum_{i}\frac{\partial}{\partial x_i}(a_i\,p) + \frac12\sum_{i,j}\frac{\partial^2}{\partial x_i\partial x_j}[(\sigma\sigma^T)_{ij}\,p] + \varepsilon^\alpha\int_{\mathbb{R}^n\setminus\{0\}} [p(x+y)-p(x)]\,\nu_\alpha(dy)$$

where $\nu_\alpha$ specifies the Lévy measure. Specializations (e.g., setting $\varepsilon=0$ or omitting the second derivative term) yield pure Brownian or pure Lévy regimes. This mathematical framework is fundamental across applications in stochastic physics, uncertainty propagation, and statistical modeling (Chen et al., 2020, Savaliya et al., 26 Oct 2025).

2. Physics-Informed Neural Network Parameterizations

Key unknowns—probability density, drift, and diffusion coefficients—are parameterized by feed-forward neural networks. For instance, the PDF is approximated as $\tilde p(x,t;\theta) = \ln(1+\exp(f(x,t;\theta)))$, where $f$ is a multi-layer network with smooth activations (tanh, softplus) (Chen et al., 2020). Drift terms may be represented by parametric (e.g., polynomial) expansions or neural networks, and diffusion parameters can be learned jointly.

For stochastic ODEs, architectures such as the Noise-Augmented State Predictor (NASP) include the realized noise as an explicit input; for example, a 3D input $(v, w, \sigma\,\eta)$ is mapped to predicted states (e.g., next time-step values) (Savaliya et al., 26 Oct 2025). Variational auto-encoders can also be used, encoding observed samples into a latent representation and generating realizations constrained by physics-based losses (Zhong et al., 2022).

3. Loss Structures and Stochastic Perturbation Mechanisms

The defining feature of these schemes is the integration of stochastic sample-based information directly into the loss. Typical total losses are sums of weighted components: $$L_{\text{total}} = \tau\,L_{\text{PDE}} + \sum_i L_{{\rm data},i} + L_{{\rm BC/IC}}$$ with

• $L_{\text{PDE}}$: residual of the governing stochastic PDE/SDE enforced on collocation points via mean-squared error,
• $L_{{\rm data},i}$: sample-based mismatch, often using a variational Kullback–Leibler divergence,
• $L_{{\rm BC/IC}}$: penalties for boundary or initial conditions (Chen et al., 2020).

For discrete realizations $\{X_{t_i}^{(j)}\}_{j=1}^{n_i}$, the data loss is cast as

$$L_{{\rm data},i} = -\frac{1}{n_i}\sum_{j}\ln\tilde p(X_{t_i}^{(j)}, t_i) + \int_D \tilde p(x, t_i)\,dx$$

enforcing maximum-likelihood consistency of the inferred PDF (Chen et al., 2020). In more specialized settings, escape dynamics and stochastic resonance are implemented as barrier-based constraints arising from Kramers' theory, e.g., matching timescale log-exponents to potential barriers $$\frac12\sigma^2\log\epsilon^{-1} \approx \Delta U(w,a)$$ (Savaliya et al., 26 Oct 2025).

Gradient-free implementations of such schemes exploit stochastic projection or local Monte Carlo to estimate spatial and temporal derivatives, obviating the need for automatic differentiation; the discrete SP-gradient is formed as a least-squares regression over randomly sampled local neighbor points (N et al., 2022).

4. Stochasticity Treatment and Types of Perturbations

Stochastic perturbations arise via several mechanisms:

• Direct use of samples: Empirical data from stochastic simulations (e.g., SDE particle snapshots) are used in variational divergence losses (Chen et al., 2020).
• Noise augmentation: Realized noise increments are input features for state-predictor architectures (Savaliya et al., 26 Oct 2025).
• Random location perturbation: Near-identity diffeomorphic transformations map state variables at each step, resulting in SPDEs whose stochastic forcing embodies location uncertainty (Zhen et al., 2022).
• Stochastic finite differences: Local directional derivatives are estimated via weighted random sampling in the input domain (N et al., 2022).

The choice and calibration of perturbation mechanism depend on the modeling context, data structure, and the type of invariants to be preserved (e.g., conservation laws, energy, mass).

5. Theoretical Underpinnings and Stability Guarantees

Physics-informed stochastic perturbation schemes are supported by rigorous theoretical foundations:

• Variational coercivity and Sobolev-embedding: Guarantee that sufficiently regular PINN solutions, constrained by physically modeled perturbations, possess pointwise error bounds proportional to the residual loss (e.g., $\|u_\theta - u^*\|_{L^\infty}$ controlled by a Sobolev norm of the loss) (Katende, 16 Jun 2025).
• Deterministic perturbation sensitivity: Linear bounds ($C_{\rm pert}\,\varepsilon$) quantify the maximal loss change due to bounded stochastic perturbations in the network output (Katende, 16 Jun 2025).
• Concentration inequalities: McDiarmid-type estimates yield explicit probabilistic bounds on deviations of empirical PINN losses due to stochastic sampling, facilitating principled choices of batch size $N$ and error budgets (Katende, 16 Jun 2025).
• Universality and regularization: Feed-forward networks with smooth activations are universal approximators; the PDE-residual and sample-based losses act as strong regularizers against overfitting, particularly for sparse-data regimes (Chen et al., 2020).

6. Numerical Implementation and Algorithmic Procedures

Typical training workflows integrate neural parameter updates, stochastic data sampling, and PDE residual evaluation. For inverse stochastic problems, an illustrative pseudocode is (Chen et al., 2020):

for iter in range(M): # 1. Sample residual points (x_j, t_j) ~ Uniform(domain) # 2. Compute PDE residuals r_j = ∂_t p(x_j, t_j; θ) - A^*[p](x_j, t_j; θ, ...) # 3. For each snapshot, sample data minibatch for i in snapshots: X_samples = select_subset(X_ti) # 4. Compute data misfit L_data += ... L_total = τ * L_PDE + L_data + L_BC/IC # 5. Backpropagate/update

For gradient-free SP-PINN, spatial/temporal derivatives are replaced by stochastic projection gradients evaluated by local neighbor sampling (N et al., 2022). Conservation-preserving location-uncertainty schemes require evaluating pullbacks and ensuring invariance for selected differential forms (Zhen et al., 2022).

7. Applications, Performance, and Extensions

Physics-informed stochastic perturbation schemes have demonstrated efficacy across a variety of applications:

• Inverse Fokker–Planck reconstruction: Accurate recovery of multidimensional PDFs and drift coefficients with sparse data; e.g., in the 5D Brownian setting, parameter error $<5\%$ after a few $10^4$ iterations (Chen et al., 2020).
• Stochastic resonance and rare event dynamics: Correct prediction of self-induced stochastic resonance (SISR) phenomena in FitzHugh–Nagumo systems; test error reductions of $\sim 43\%$ with physics constraints (Savaliya et al., 26 Oct 2025).
• Uncertainty quantification in Bayesian PINNs: Multi-replica stochastic gradient MCMC methods efficiently explore multimodal posteriors and accelerate convergence with hybrid fidelity solvers (Lin et al., 2021).
• Non-smooth and irregular domains: SP-PINN exhibits strong accuracy and robustness, especially with non-differentiable activations and on domains with sharp features or discontinuities (N et al., 2022).
• Covariance inflation and invariant-preserving SPDEs: Location-uncertainty schemes enforce conservation of critical invariants (mass, energy, helicity) at each stochastic perturbation step, with generalization to arbitrary field theories (Zhen et al., 2022).

A summary of methodological advantages and empirical outcomes is given below:

Scheme	Perturbation Type	Key Outcomes
PINN + KL-div loss (Chen et al., 2020)	Stochastic sample via data loss	Accurate PDF/drift inference; superior to kernel methods
NASP SDE PINN (Savaliya et al., 26 Oct 2025)	Input-level noise perturbation	Coherence curve recovery; improved generalization
SP-PINN (N et al., 2022)	Stochastic projection gradients	Robust to discontinuities, non-smooth activations
Location-uncertainty SPDE (Zhen et al., 2022)	Random diffeomorphism	Invariant conservation at every step; unifies SALT/LU form
Multi-replica SGLD (Lin et al., 2021)	Langevin dynamics with replica swaps	Efficient Bayesian PINN sampling; reduced computation

• "Solving Inverse Stochastic Problems from Discrete Particle Observations Using the Fokker-Planck Equation and Physics-informed Neural Networks" (Chen et al., 2020)
• "Self-induced stochastic resonance: A physics-informed machine learning approach" (Savaliya et al., 26 Oct 2025)
• "Stability Analysis of Physics-Informed Neural Networks via Variational Coercivity, Perturbation Bounds, and Concentration Estimates" (Katende, 16 Jun 2025)
• "Physically Constrained Covariance Inflation from Location Uncertainty" (Zhen et al., 2022)
• "Stochastic projection based approach for gradient free physics informed learning" (N et al., 2022)
• "PI-VAE: Physics-Informed Variational Auto-Encoder for stochastic differential equations" (Zhong et al., 2022)
• "Stochastic Physics-Informed Neural Ordinary Differential Equations" (O'Leary et al., 2021)
• "Multi-variance replica exchange stochastic gradient MCMC for inverse and forward Bayesian physics-informed neural network" (Lin et al., 2021)