Physics-Infused Architectures

• Physics-infused architectures are computational frameworks that embed governing equations and conservation laws into ML models, ensuring physical consistency and robust generalization.
• They blend embedded physics modules, constrained loss functions, and symmetry-aware layers to faithfully enforce inductive biases and reduce extrapolation errors.
• These frameworks enhance data efficiency, interpretability, and reliability, making them crucial for advancing scientific computing and engineering design.

Physics-infused architectures are computational frameworks that integrate physical knowledge—such as governing equations, symmetries, or domain specific inductive biases—directly into the structure or training of machine learning models. These architectures span a spectrum from hard-coded physics modules embedded within neural networks, to data-driven models with emergent physical consistency, to architectures leveraging symmetry group equivariance, and even to self-organizing systems governed by local conservation laws. The core objective is to improve extrapolation, generalizability, interpretability, and physical reliability, especially in scientific and engineering domains where adherence to physical law is non-negotiable.

1. Foundational Principles and Motivations

Physics-infused architectures are motivated by the fundamental limitations of purely data-driven machine learning, particularly when training data are sparse or do not adequately explore extrapolative regimes. Standard neural networks may violate basic physical laws, yield nonphysical outputs, or generalize poorly, especially in computational science, engineering, and physical modeling tasks. By contrast, integrating prior knowledge—in the form of partial differential equations (PDEs), conservation laws, symmetry principles, or reduced-order models—can inductively bias a model toward solutions that remain physically admissible on unseen, out-of-distribution inputs (Wiesner et al., 17 Sep 2025, Iqbal et al., 2022, Natale et al., 2021, Bogatskiy et al., 2022).

Two canonical motivations are:

• Generalization and Extrapolation: Embedding physics constrains the hypothesis space, allowing architectures to perform robustly on long-range predictions, new boundary conditions, and even novel physical regimes with little or no retraining (e.g., zero-shot generalization in GPhyT (Wiesner et al., 17 Sep 2025), fourfold lower extrapolation errors in OPTMA-Net (Iqbal et al., 2022)).
• Interpretability and Inductive Bias: Hard coding invariants, symmetries, or conservation laws yields interpretable latent representations and layer outputs, enhancing trust in resultant models for scientific workflows (Nautrup et al., 2020, Natale et al., 2021, Bogatskiy et al., 2022).

2. Core Categories and Architecture Types

Physics-infused architectures can be classified into several overlapping but distinct categories:

A. Data-to-Physics Blending via Embedded Physics Models

These frameworks integrate physics-based operators (e.g., analytical, reduced-order, or low-fidelity simulators) as fixed or partially tunable network modules, often inside neural pipelines. Examples include:

• OPTMA/OPTMA-Net: A transfer neural network predicts parameters that are fed directly into a differentiable partial physics model (e.g., a monopole-based acoustic field), yielding better extrapolation, interpretability, and data efficiency (Iqbal et al., 2022).
• PIROM: Embeds lumped-parameter physical models with data-driven correction terms (via Mori-Zwanzig formalism) into a reduced-order ODE architecture, maintaining physical plausibility across material and boundary condition changes while achieving sub-1% error (Venegas et al., 28 May 2025).
• Physics Encoded Residual Architectures (PERNN): Hard-codes physics operators (e.g., Euler-Lagrange equations for robot dynamics, pure-pursuit steering for control) as static blocks with residual neural compensation, facilitating reliable generalization and fast training convergence (Zia et al., 2024).
• PhysicsNAS: Uses neural architecture search over a space containing physics blocks (as inputs, operations, or residuals), enabling the learning of architecture optimal for the known accuracy of the physics model and data regime (Ba et al., 2019).

B. Physics-Informed Neural Networks (PINNs)/Constrained Losses

Here, the governing equations (typically PDEs) enter directly as components of the loss function:

• PINNs: Networks are trained to minimize the sum of data mismatch and physics residual losses, enforcing solution adherence to PDE constraints at collocation points (2402.02711).
• Physics-Constrained Neural Networks (PCNN): Train not only on supervised data but also on consistency with a physics-based surrogate, as in composite materials (forward-inverse weave mapping) (Feng et al., 2022), or to enforce monotonic thermodynamic coupling in building thermal modeling (Natale et al., 2021).
• Multi-LSTM Physics-Informed Architectures: Embed governing ODEs, state-dependence, and hysteretic relationships as losses within recurrent networks for structural dynamics (Zhang et al., 2020).

C. Emergent Physics in Purely Data-Driven Architectures

Certain architectures aim for "emergent" rather than imposed physical structure, relying on inductive biases such as self-attention or explicit symmetry group equivariance:

• GPhyT (General Physics Transformer): A transformer trained on diverse simulation fields learns to infer governing dynamics via in-context learning, achieving cross-domain generalization and outperforming specialized neural operators (Wiesner et al., 17 Sep 2025).
• Symmetry Group Equivariant Networks: Layers are constructed to be exactly equivariant to physical symmetry groups (e.g., translation, rotation, Lorentz, gauge), yielding dramatically improved parameter efficiency, robustness to domain shift, and physical consistency (Bogatskiy et al., 2022).

D. Local, Self-Organizing, and Physics-Embedded Update Rules

Some models dispense with global loss entirely, instead letting local rules encode physics:

• Hebbian Physics Networks (HPN): Nodes update states and weights based solely on violations of conservation laws at each link; self-organization leads to physically consistent global solutions without gradient-based optimization (Auti et al., 1 Jul 2025).
• Equilibrium Propagation, Variational Hardware-Inspired Networks: Learning is realized as local parameter updates in physical (neuromorphic, electrical, mechanical) substrates minimizing an energy functional, operationalizing variational principles at the architecture level (Scellier, 2021).

3. Mathematical and Algorithmic Foundations

Physics-infused architectures codify physical laws through a range of mathematical mechanisms:

• Explicit Equation Embedding: Hard constraints via analytically encoded operators (e.g., lumped-capacitance ODEs, Hamiltonians, RC networks) within forward passes or residual modules (Zia et al., 2024, Natale et al., 2021, Venegas et al., 28 May 2025).
• Loss-Based Regularization: Soft constraint via loss augmentation, e.g., adding the squared norm of the PDE residual (PINNs), or using symmetry penalty terms for even/odd decomposition (2402.02711, Barber et al., 2021).
• Physics as Inductive Bias via Symmetries: Architecture designed for group equivariance using representation theory (irreps, group-convolution, tensor field networks), guaranteeing by construction that outputs co-vary with physical group actions (Bogatskiy et al., 2022).
• Local Residual-Driven Adaptation: Local rules where weights are adapted in response to residuals on physical conservation, establishing a correspondence with entropy production in non-equilibrium thermodynamics (Auti et al., 1 Jul 2025).
• Neural Tangent Kernel and Preconditioning: Optimization of PINN architectures using NTK analysis and preconditioned hidden layers (e.g., Gaussian activations, row-equilibration) to enhance PDE-task trainability (2402.02711).

4. Performance Benchmarks and Comparative Evaluation

Physics-infused architectures have been systematically benchmarked against purely data-driven surrogates, black-box NNs, and traditional numerical solvers:

Architecture	Generalization Regime	Extrapolation Error	Data Efficiency	Physical Consistency	Training Overhead
GPʰʸT (Wiesner et al., 17 Sep 2025)	Cross-physics, BC	≤29× lower than FNO	“Train once, deploy anywhere”	Emergent	High (XL variant)
OPTMA/Net (Iqbal et al., 2022)	Acoustic field, UAV	~4× lower than ANN	No extra solver runs needed	Guaranteed (by model)	Moderate
PIROM (Venegas et al., 28 May 2025)	Multilayer TPS, OOD BC	<1% vs. 7–15% (NODE)	Accuracy w/ OOD data	Yes (Markovian origin)	High (offline)
PCNN (Natale et al., 2021)	Building temp., 3-day	32–41% lower than RC	Suppresses overfitting	Provable monotinicity	Comparable to NN
PERNN (Zia et al., 2024)	Robot/Vehicle control	40× lower MSE than DeLaN	Matches FCNN-large with 1/140 parameters	Embedded operators	Two-phase
PINNs (2402.02711)	Test PDEs (Burgers, N–S)	Orders-of-mag lower with Gaussian–EqPINN	See above	By construction	Moderate
Symmetry Eq. (Bogatskiy et al., 2022)	LHC jets, mol. potentials	Out-of-domain error flat	10–100× fewer samples needed	Exact equivariance	2–5×/epoch

These quantitative results are context specific, but underline the typical gains (sometimes order-of-magnitude reductions in extrapolation error, 10–100× higher parameter/data efficiency, and stable enforcement of physical constraints) achieved when physics is systematically infused at the architectural level.

5. Limitations, Trade-Offs, and Design Considerations

Despite substantial successes, several limitations and open design considerations remain:

• Physics Model Accuracy: Embedding a coarse, incomplete, or invalid physics block may limit the attainable accuracy or even mis-regularize the network. Several frameworks (PhysicsNAS, PERNN) mitigate this by allowing learned residuals or adaptive fusion (Ba et al., 2019, Zia et al., 2024).
• Scalability and Computational Cost: Large physics-infused transformers (e.g., GPʰʸT-XL) incur high inference and training costs. Some techniques (Gaussian activation PINNs, equilibrated architectures) trade increased per-step cost for order-of-magnitude gains in convergence (Wiesner et al., 17 Sep 2025, 2402.02711).
• Generalizability Across Physics Domains: While general-purpose architectures such as GPhyT and PIROM obtain cross-domain success, most current models remain confined to the domains for which their internal physical representations are tailored (e.g., fluids, thermal, robot dynamics). Extension to multi-physics or multi-modal scientific problems remains an open challenge (Wiesner et al., 17 Sep 2025).
• Hyperparameter Sensitivity: Local-update networks (HPNs) and PINN optimizations can be sensitive to learning rates, weight-decay, and distribution of physical residuals (Auti et al., 1 Jul 2025, 2402.02711).
• Development Overhead: Symmetry-equivariant architectures require explicit encoding of group representations and can impose significant software engineering effort, especially for noncompact or nontrivial groups (Bogatskiy et al., 2022).

6. Implications, Outlook, and Future Directions

Physics-infused architectures are converging toward several potential future trajectories:

• Physics Foundation Models: Models like GPhyT point toward a paradigm shift—single, large-scale, data-driven networks capable of simulating heterogenous physics domains through in-context learning, eliminating the need for narrow, equation-specific networks (Wiesner et al., 17 Sep 2025).
• Automated Architecture Search and Adaptive Infusion: Methods like PhysicsNAS pave the way for automatic architecture design that tailors the degree, location, and type of physics-infusion to the quality of prior models and dataset, promising flexible, robust pipelines for hybrid physical-ML modeling (Ba et al., 2019).
• Integration of Equivariance and Local Adaptation: Combining group-equivariant designs with local residual-driven adaptation (as in HPNs) could yield architectures both globally robust and locally interpretable (Auti et al., 1 Jul 2025, Bogatskiy et al., 2022).
• Hardware and Neuromorphic Integration: Physics-infused algorithms such as equilibrium propagation offer recipes for implementing scalable learning directly on physical or neuromorphic substrates, promising energy-efficient, in-memory computation (Scellier, 2021).
• Cross-Domain and Multi-Scale Models: Extending current frameworks to embrace 3D physics, multi-physics coupling, and variable-resolution architectures is a subject of ongoing research (Wiesner et al., 17 Sep 2025).
• Standardization and Benchmarks: Establishing community-wide benchmarks, metrics, and standardized OOD tests is recognized as a key requirement for progress (Bogatskiy et al., 2022).

Physics-infused architectures thus provide a rigorous, extensible framework for harnessing physical prior knowledge in machine learning, unlocking superior generalization, reliability, and physical interpretability across computational science and engineering. The broad spectrum of design patterns and successful empirical results suggest that infusing physics at the architecture level will remain a driving principle in the next generation of scientific machine learning.