PIWSL: Physics-Informed Weakly Supervised Learning

• PIWSL is a framework that embeds physical laws as prior constraints in deep learning, enabling effective learning in data-scarce regimes.
• It leverages physics-based loss functions—such as PDE residual penalties and Taylor-consistency losses—to regularize training with weak or indirect supervision.
• PIWSL has demonstrated success across applications like surrogate modeling, interatomic potentials, and inverse problems, highlighting its versatility in scientific deep learning.

Physics-Informed Weakly Supervised Learning (PIWSL) is a paradigm that integrates domain-specific physical knowledge as explicit constraints or priors in machine learning, without relying on large labeled datasets. By enforcing known (or surrogate) physical laws at the output or intermediate stages of a neural network, PIWSL regularizes model training through “soft” physics-based losses, weak direct supervision, and self-supervision, thus enabling high-fidelity learning even in data-scarce regimes and where labels are unavailable or only partially known. This methodology has demonstrated substantial gains across PDE-governed surrogate modeling, interatomic potentials, scientific inverse problems, and representation learning in dynamical systems.

1. Principle and Framework of PIWSL

PIWSL operates by encoding physical knowledge—frequently in the form of partial differential equations (PDEs), energy conservation laws, or transformation-invariant constraints—into learnable loss functions. Weak supervision denotes the use of partial, indirect, or noisy labels (or, in some settings, completely unlabeled data), with physics-based consistency serving as the main learning signal.

A canonical PIWSL instance is the physics-informed loss for the steady-state heat equation, where the absence of labeled data is mitigated via a loss term corresponding to the Laplacian residual of the network’s output. For temperature prediction $T(x, y)$ on a 2D grid, the loss is constructed by convolving the output with a finite-difference Laplacian kernel:

$$K = \begin{bmatrix} 0 & -1 & 0 \ -1 & 4 & -1 \ 0 & -1 & 0 \end{bmatrix}$$

The total loss is then:

$L_{\rm phys}(\hat{Y}) = \|K * \hat{Y}\|_2^2$

with no supervision from ground-truth $(u^*, T^*)$ pairs—only the requirement that predicted fields satisfy the PDE and imposed boundary conditions (Sharma et al., 2018).

In atomistic learning, PIWSL regularization includes Taylor-consistency and conservative-force losses based on first-order energy expansions and the curl-free property of force fields. For example, the Taylor-consistency loss uses energetic predictions under small perturbations as weak targets derived from gradients:

$$L_{\rm PITC}(\mathcal{S};\theta) = \ell\left(E(\mathcal{S}_r;\theta), E(\mathcal{S};\theta) - \sum_i \langle r_i, \mathbf{F}_i(\mathcal{S};\theta) \rangle \right)$$

(Takamoto et al., 2024).

In operator learning where the true governing PDE is unknown, pseudo-physics-informing builds a surrogate PDE model $\phi$ using only simple differential operators and trains jointly with a neural operator via alternating updates. This regularizes the primary operator by penalizing deviations from the learned local physics (Chen et al., 4 Feb 2025).

The key constituents of PIWSL thus include:

• Physics-informed loss, directly encoding physical consistency;
• Weak or indirect supervision via partial labels or physical projections;
• Physics-augmented or surrogate models when ground-truth physics is absent.

2. Core Methodologies and Loss Construction

The construction of PIWSL models is dictated by the type of physical knowledge and data available. Several representative strategies include:

• Direct PDE Residual Penalty: For steady-state PDEs, the physics loss is the squared norm of the residual field obtained via a fixed stencil (e.g., Laplacian) convolved with the prediction. Multi-scale residuals and progressive weighting can improve optimization and avoid trivial solutions at high resolution (Sharma et al., 2018).
• Physics-Constrained Projections: When only indirect labels $H$ related to the ground-truth $Y$ via a known mapping are available, outputs are projected onto the physics-feasible subspace determined by $H$. The loss penalizes deviation from this subspace using the orthogonal projector $P = A(A^\top A)^{-1} A^\top$, where $A$ encodes the physical relationship (Chen et al., 2020).
• Taylor Expansion and Conservative-Force Losses: For interatomic potentials, physically-consistent energy extrapolation and conservative-force properties yield two auxiliary losses: (a) Taylor-expansion (PITC) penalizes inconsistency between perturbed energies and local gradients, (b) conservative-force (PISC) penalizes path-inconsistency in energy predictions (Takamoto et al., 2024).
• Surrogate Physics Networks: In operator learning settings lacking closed-form PDEs, a surrogate PDE $\phi$ is constructed as a small neural network on local finite-difference features, then used as a constraint in training the larger operator model (Chen et al., 4 Feb 2025).
• Weak Supervision via Partial or Interval Labels: In settings such as physically-interpretable latent variable learning, only interval or indirect supervision is available. A Kullback-Leibler divergence is used to match the latent posterior to a Gaussian prior parameterized by the interval, enforcing that a subset of latent variables align with physical quantities within prescribed tolerance (Mao et al., 2024).

The losses are typically differentiated and optimized in end-to-end fashion, either as pure physics-informed objectives or combined (with tunable weighting) with classical supervised or self-supervised losses.

3. Network Architectures and Training Strategies

PIWSL architectures are frequently tailored to combine standard deep architectures with modules encoding hard-wired or differentiable physical processes.

• U-Net-style Fully Convolutional Networks: Applied to PDE-solved tasks with multi-scale physics loss, ensuring resolution-invariance and leveraging skip connections for efficient learning (Sharma et al., 2018).
• Convolutional-Encoder plus Physics Decoder: Used in hybrid approaches (e.g., rock- and wave-physics informed models) where an image or sequence is mapped to a latent state (e.g., porosity), then forward-simulated through a differentiable physics block; gradients propagate through this pipeline (Vashisth et al., 2022).
• Alternating Update Schemes: Surrogate-physics-informed operator learning alternates between refining the main neural operator parameters and the surrogate PDE, exploiting Monte Carlo draws over the input space to stabilize the pseudo-physics loss (Chen et al., 4 Feb 2025).
• Self-supervised and Physics Priors in Representation Learning: For visual world modeling, a convolutional encoder maps images to structured latent spaces with designated physically-interpretable channels, enforced by interval priors and embedded known dynamics, with reconstructive and KL-based interval-consistency terms (Mao et al., 2024).

Common training enhancements include batch normalization or normalization of projected outputs (to prevent collapse), curriculum learning (for multi-scale losses), dropout to prevent overfitting, and early stopping based on validation loss. Empirical results show substantial error reductions when compared to purely supervised or non-physics-informed baselines, particularly under data scarcity.

4. Representative Applications and Quantitative Performance

PIWSL has been effectively deployed in a range of scientific and engineering contexts. Significant results are documented for each application:

Domain/Problem	Model/Approach	Physics-Informed Loss Type	Notable Outcomes
2D Heat Equation	U-Net + Multi-scale PDE	Laplacian residual (multi-scale)	<1.5% mean per-pixel error up to 1024×1024 (Sharma et al., 2018)
Interatomic Potentials	SchNet/PaiNN/others	Taylor-consistency + conservative-force	Energy RMSE 10–60% drop, force error 5–20% drop, robust MD flows (Takamoto et al., 2024)
Well Log Prediction	LSTM/Batch norm	Physics-projection via indirect labels	MSE≈0.065 (no direct labels), shape and trend preservation (Chen et al., 2020)
Operator Learning (unknown PDE)	DeepONet/FNO + Surrogate PDE	Pseudo-physics surrogate PDE	Error reduction up to 95% vs. vanilla operator learning, rivaling true-PDE-informed PINOs (Chen et al., 4 Feb 2025)
Petrophysical Inversion	CNN + physics decoder	Self-supervision+weak labeled logs	Porosity RMSE 0.05 vs 0.043 (fully supervised); seismic NRMS ≈ 0.004 (Vashisth et al., 2022)
Visual State Representation	Conv-AE + latent physics	Interval-based KL + known dynamics	20–50% MAE reduction versus standard VAE/LSTM, high image SSIM (Mao et al., 2024)

This diversity underscores the generality of PIWSL; it applies wherever physics can inform or constrain outputs, even with minimal supervision.

5. Limitations, Best Practices, and Extensions

Limitations of PIWSL approaches are often dictated by:

• Applicability to steady-state or translation-invariant physics (e.g., Laplace PDE, not yet time-dependent problems in the original form) (Sharma et al., 2018).
• Reliance on availability or learnability of appropriate surrogate physics in the absence of true governing laws, which can limit physical fidelity (Chen et al., 4 Feb 2025).
• Difficulty with scale-ambiguity or representation collapse unless supervised with sufficient diversity or normalized projection (Mao et al., 2024).
• Potential performance degradation as physics priors become weaker (e.g., wide intervals in interval-based supervision).

Best practices include:

• Multi-scale or curriculum training to avoid trivial or degenerate solutions when loss weighting is highly imbalanced.
• Use of small perturbations, first-order Taylor consistency, and random direction sampling in energy/force learning.
• Keeping surrogate physics networks lightweight and the differential operator basis simple (first- or second-order) to prevent overfitting.
• Projection normalization and sign-correction where the physics-prior subspace is degenerate (Chen et al., 2020).
• Careful tuning of loss weights, and validation under increasing data sparsity.

Potential extensions actively investigated include:

• Explicit support for time-dependent PDEs via temporal residuals or inclusion of time as an input channel (Sharma et al., 2018).
• Surrogate-physics learning for more complex, nonlinear or multi-physics systems.
• Adaptive or active strategies for tightening weak supervision, e.g., interval active learning (Mao et al., 2024).
• Multi-fidelity PIWSL combining analytic physics, surrogate models, and experimental data (Chen et al., 4 Feb 2025).
• Learning of unknown stencils or physical laws directly from high-quality solution libraries (Sharma et al., 2018).

6. Relation to and Distinction from Other Paradigms

PIWSL is distinguished from fully supervised learning by its capability to operate with minimal or absent ground-truth labels, leveraging physics consistency as the main learning driver. Compared to strict self-supervised learning, PIWSL offers stronger generalization by tying representations to established scientific laws rather than statistical patterns alone.

Relative to PINNs and physics-informed neural operators, PIWSL relaxes the requirement for known exact PDEs, instead allowing for the enforcement (or even learning) of approximate or partial physical constraints—pseudo-physics—directly from data (Chen et al., 4 Feb 2025). It also accommodates domains with indirect or partial supervision, using projection or interval priors, and is compatible with hybrid architectures for forward-inverse scientific inference (Vashisth et al., 2022).

This flexibility has positioned PIWSL as an essential framework for scientific deep learning in low-data and partially observed settings, and as a methodological bridge between empirical modeling and first-principles simulation.