Non-Intrusive Surrogate Modeling

• Non-intrusive surrogate modeling is a data-driven approach that replicates high-fidelity simulations by decoupling offline sampling from regression mapping without altering governing equations.
• It utilizes techniques like autoencoders, POD, and Gaussian processes to compress and map high-dimensional data, achieving speed-ups up to 1000× with reconstruction errors below 1.5%.
• Applications include uncertainty quantification, optimization, and real-time design exploration in fields such as fluid dynamics, structural mechanics, and circuit analysis while preserving physical constraints.

A non-intrusive surrogate model is a machine-learning-based, purely data-driven reduced model that replicates the input–output behavior of a high-fidelity simulator (often a black-box PDE solver or numerical code) without requiring modification, access, or embedding into the original model’s governing equations or discretization routines. Non-intrusive surrogate modeling subsumes advanced techniques for predictive modeling, uncertainty quantification, optimization, and design exploration in engineering, physics, and data sciences. The approach is characterized by its separation of dataspace sampling (offline, high-fidelity queries) from the construction of regression or interpolation maps (offline/online, low-cost evaluation).

1. Mathematical Formulation and Core Principles

A non-intrusive surrogate model approximates the solution map of a parametrized nonlinear time-dependent PDE system: $$\partial_t u(x,t;\mu) + \mathcal{N}[u(x,t;\mu)] = f(x,t;\mu), \quad x\in\Omega,\; t\in[0,T],\; \mu\in\Theta\subset\mathbb{R}^n,$$ where $u$ is the field of interest, $\mu$ is the vector of parameters (physical properties, loads, etc.), and $\mathcal{N}$ is a nonlinear spatial differential operator. Standard approaches involve discretizing the PDE (e.g., finite elements or volumes $\rightarrow$ $d$ spatial degrees of freedom), and computing solution trajectories $U(\mu) \in \mathbb{R}^{d\times N_t}$ for sampled $\mu$.

Non-intrusive surrogates construct $\widehat{U}(\mu)$ using dimensionality reduction and regression. A general paradigm employs:

• A nonlinear feature extractor (autoencoder, POD, kernel PCA) to compress $U(\mu)$ into low-dimensional latent variables $z(\mu)\in\mathbb{R}^\ell$.
• A regression/interpolator $\Phi:\Theta\to\mathbb{R}^\ell$ (neural network, Gaussian process, RBF, random feature expansion) to map $\mu \mapsto z(\mu)$.
• A decoder reconstructing $U(\mu)$ from the latent representation.

This architecture bypasses the need to re-solve the equations at each query, leveraging stored solution libraries for training (Nikolopoulos et al., 2021, Kneifl et al., 2023, Kehls et al., 2 Sep 2025). The choice of compression and regression is problem-dependent (see Table below).

Compression	Regression	Representative Paper
CAE (Conv Autoenc)	FFNN (Dense)	(Nikolopoulos et al., 2021)
PCA/KPCA/AE/VAE	Gaussian Process	(Kneifl et al., 2023)
POD (SVD)	RBF Interpolator	(Khowaja et al., 2021)
POD	Kernel Shallow NN	(Kapadia et al., 2023)
Hammerstein Table	LTI SysID	(Hanson et al., 2024)
Sparse Random Feat	LASSO	(Herold et al., 2022)

2. Model Architectures and Algorithms

Convolutional Autoencoder + FFNN (CAE-FFNN):

The high-dimensional solution matrix $U(\mu)$ is encoded as $z=E(U)\in\mathbb{R}^\ell$ using stacked 1D convolutional layers, average-pooling, and dense layers (time-direction only for spatiotemporal problems). The decoder reconstructs the field. FFNN regression $\Phi(\mu)$ (4–6 layers, ReLU, linear output) is trained to match $z=E(U(\mu))$ for each offline $\mu$. Online evaluation at new $\mu$ computes $z = \Phi(\mu)$, $U = D(z)$. Reconstruction error 0.5–1.5% on Burgers’ and shear-wall benchmarks, with 80×–1000× speed-up (Nikolopoulos et al., 2021).

POD–RBF and similar low-dimensional regressors:

SVD/POD identifies principal modes of solution snapshots, projecting $U\approx\Phi_r a$ ($\Phi_r$ basis, $a$ coefficients). RBF, kernel-based, or shallow NN interpolation captures $a(\mu)$. Latin hypercube sampling ensures stratification. Typical relative errors: $\sim$0.1–1% for nonlinear ODE/PDE optimizations (Khowaja et al., 2021, Kapadia et al., 2023).

Non-intrusive Operator Inference (OpInf):

For systems with parametric operators, high-fidelity data is projected onto reduced coordinates; Markovian operator dynamics (e.g. linear–quadratic structure) are inferred via minimum-norm least-squares over projected states and derivatives. This yields parametric surrogate operators respecting conserved quantities (Yıldız et al., 2021).

Autoencoder-based Multi-field & Force-augmented Models:

Encoders for both field and boundary variables compress to shared (possibly concatenated) latent spaces, supporting direct field-force and coupled multi-physics predictions (Kehls et al., 2 Sep 2025).

Sparse Random Feature Surrogates:

Kernel PCA reduces dimensionality; sparse random feature expansions (low $q$-sparse features, LASSO fit) yield data-efficient interpolators outperforming PCE and NNs under limited sampling (Herold et al., 2022).

3. Construction, Sampling, and Training Protocols

The typical workflow involves:

1. Offline Sampling: Draw $N$ parameter values $\{\mu_i\}$ via Monte Carlo, Latin-Hypercube, or adaptive (active-learning) strategies, covering the domain $\Theta$. For each $\mu_i$, solve the high-fidelity model to obtain $U_i$ (solution snapshots, response quantities).
2. Compression: Learn encoder $E$ and decoder $D$ minimizing reconstruction error $\mathcal{L}_{AE}$. Choices include SVD/POD, kernel PCA, autoencoders, or physical symmetry-enforced representations. Latent dimensions determined by "retained energy" or cross-validation.
3. Regression/Interpolation: Train regression $\Phi$ from $\mu_i$ to $z_i=E(U_i)$ with neural networks (dense FFNNs), Gaussian process regression, RBFs, or LASSO/LOS-RFE expansions. Loss functions quantify misfit in latent space.
4. Online Prediction: For new $\mu$, predict $z=\Phi(\mu)$ and reconstruct $\widehat{U}=D(z)$.

Error metrics include normalized matrix error ($\epsilon=\|U_{FOM}-U_{SUR}\|_2/\|U_{FOM}\|_2$), mean element/node errors, and physically relevant quantities (stress, force, statistical moments). Models are validated by comparison to direct simulation, cross-validated on hold-out parameters and spatiotemporal domains (Nikolopoulos et al., 2021, Kneifl et al., 2023).

4. Applications and Performance Benchmarks

Non-intrusive surrogate modeling is directly applicable to:

• Uncertainty Quantification (UQ): Efficient Monte Carlo sampling, surrogate-based polynomial chaos, and sensitivity analysis. Surrogate-predicted mean/variance often stays within 1–3% of full-model values with bulk speed-ups of 10²–10³× (Nikolopoulos et al., 2021, Ye et al., 2020, Hijazi et al., 2019).
• Optimization and Inverse Problems: Surrogate-driven optimization (using RBF–POD or AE–GP), with robust performance under nonlinearity. Multi-query optimization achieves 3–4× faster solution time in tested scenarios (Khowaja et al., 2021, Kneifl et al., 2023, Herold et al., 2022).
• Design Exploration and Control: Real-time computation, interactive parameter sweeps, and integration into digital twins for design feedback (Kehls et al., 2 Sep 2025).
• Multi-physics and Shape-varying Domains: Efficient treatment of geometric variation via parent-domain mappings and landmark-based coordinates enables fast surrogates for shape optimization and patient-specific simulations (Guo et al., 2022, Regazzoni et al., 2022).
• Circuit and System Identification: Hammerstein models (static nonlinearity + LTI dynamics), non-intrusively identified from DC/AC sweeps and transient experiments; supports extension to a range of analog/digital circuits (Hanson et al., 2024).

Benchmark metrics reported:

• Burgers’ equation, $N=100$, latent $\ell=8$, error 0.5–1.3% (Nikolopoulos et al., 2021).
• Shear-wall under seismic load, $N=500$, error $\approx$1.5%, speed-up ×80 (Nikolopoulos et al., 2021).
• Cook’s membrane with two-scale FE: compliance error $\approx$1%, speed-up 10³× (Guo et al., 2022).
• Crashworthiness (36-dim inputs): LOS-RFE error $\approx$1.86×10⁻², superior to PCE and NN (Herold et al., 2022).

5. Physics Preservation and Non-Intrusiveness

Non-intrusive surrogate modeling is distinguished by its agnosticism to the underlying model's physics implementation. All governing equations, discretizations, and operator details remain encapsulated within the solver at the data-generation phase. The surrogate is constructed solely from input–output samples, ensuring compatibility with legacy codes and proprietary simulation tools.

Physical structure is retained either by design (e.g., exact conservation via flux-form neural networks as in entropy-stable conservative flux form nets (Liu et al., 22 Oct 2025)) or via intelligent choice of compression and reconstruction architectures (e.g., periodicity and equilibrium satisfied in reduced bases for microstructural stress fields (Guo et al., 2022)). Constraint imposition is feasible via loss augmentation (weak or strong, e.g., PDE residuals, boundary conditions), but never via code modification.

This separation supports rapid prototyping, model-based design, and UQ in disciplines where code access is restricted.

6. Limitations, Extensions, and Future Directions

Principal limitations of non-intrusive surrogate modeling include:

• Offline cost: Solution library generation can be prohibitively expensive for high-dimensional or stiff problems.
• Hyperparameter tuning: Requires careful selection of latent dimension ($\ell$), network sizes, regularization weights, and kernel parameters to balance accuracy and generalization.
• Extrapolation risk: Surrogates may fail catastrophically outside the training domain $\Theta$—a critical risk in operational and design contexts.
• Dimensionality curse: Regression and interpolation degrade with increasing parameter dimension; solutions include sparse/active learning (Kapadia et al., 2023, Halder et al., 2019).
• Interpretability: Some deep architectures sacrifice physical interpretability compared to lumped-parameter or operator-inferred surrogates (Wang et al., 2022).

Current and proposed extensions include:

• Physics-informed Losses and Adaptive Enrichment: PINN-style residual penalization (Li et al., 10 Nov 2025), online adaptation via active learning or error-driven sample refinement (Kapadia et al., 2023, Halder et al., 2019).
• Multi-fidelity Coupling: Integration of coarse-mesh solutions into surrogate construction reduces cost and increases robustness (Nikolopoulos et al., 2021, Burnaev et al., 2017).
• Time Extrapolation via Koopman/DMD and Data Augmentation: Kernel dynamic mode decomposition for latent-space trajectory extension improves surrogate generalization in time-dependent problems (Sun et al., 2024).
• Mesh-free and Universal Geometry Models: Landmark-encoded neural surrogates accommodate complex, patient-specific and design-variant geometry without retraining (Regazzoni et al., 2022).

7. Comparative Table: Representative Non-Intrusive Surrogate Modeling Approaches

Approach	Compression	Regression	Physics Retention	Domain	Speed-up	Typical Error	Reference
CAE+FFNN	CAE	FFNN	Data-driven; Weak/Strong BC	Time-dependent PDE	80–1000×	0.5–1.5%	(Nikolopoulos et al., 2021)
POD+GPR	POD	GP	Periodicity, equilibrium	Microstructure, 2-scale	10³×	0.13–1.5%	(Guo et al., 2022)
SVD/POD+RBF/KSNN	POD/SVD	RBF/KSNN	Data-driven	ODE/PDE, UQ	3–10×	<1%	(Khowaja et al., 2021Kapadia et al., 2023)
LOS-RFE	KPCA	Sparse RF	Data-driven	Crash/UQ	–	1.9–6.2e–3	(Herold et al., 2022)
Hammerstein	Table AE	LTI SysID	DC/AC curve exact	Circuits	–	<2μA, ±0.1dB	(Hanson et al., 2024)

In summary, the non-intrusive surrogate model paradigm leverages advanced datadriven compression and regression techniques to efficiently emulate high-fidelity simulations, enabling scalable, accurate uncertainty quantification, optimization, and real-time decision support without modification to legacy model codes or loss of physical interpretability. Ongoing developments extend applicability to multi-physics, adaptive learning, and mesh-free modeling regimes.