Permutation-Invariant PINNs for Physical Systems

• Permutation-invariant physics-informed neural networks (PI-PINNs) are a framework that combines deep sets for symmetry with physics-based loss functions to enforce differential equation constraints.
• The method ensures physical laws such as acoustic reciprocity and conservation by integrating PDE/ODE residuals and permutation-equivariant layers into the training process.
• Empirical evaluations in sound field reconstruction and multi-particle dynamics demonstrate significant error reduction and improved generalization over traditional techniques.

A permutation-invariant physics-informed neural network (PI-PINN) is a neural network architecture that simultaneously enforces permutation invariance under the action of a finite symmetric group—typically the symmetric group on $N$ elements, $S_N$—and encodes physical constraints via the inclusion of partial differential equation (PDE) or ordinary differential equation (ODE) residuals in the training loss. This construction is especially pertinent in scenarios where the underlying function or physical law is symmetric in its arguments, as in acoustic reciprocity (sound field reconstruction), multi-particle dynamics, or any system of indistinguishable entities. The integration of permutation symmetry and physics-informing constraints enables both generalization across variable configurations and preservation of physically mandated symmetries and conservation laws.

1. Foundational Principles

Permutation-invariance arises in systems where inputs are unordered collections, and the target function outputs remain unchanged under element permutations. In physical modeling, this is common in multi-agent or multi-particle settings, and in sound field reconstruction where acoustic reciprocity ($P(r, s, f) = P(s, r, f)$) is required.

The neural function $f(X)$, where $X = \{x_1, ..., x_n\}$ is an unordered set, is permutation-invariant if $f(\pi X) = f(X)$ for any permutation $\pi$ of the elements. According to the "deep sets" framework, any permutation-invariant function can be universally approximated as $f(X) = \rho\left(\sum_{x\in X} \phi(x)\right)$, where $\phi$ and $\rho$ are learnable mappings, typically multilayer perceptrons (MLPs) (Chen et al., 27 Jan 2026).

The physics-informed paradigm extends conventional neural architectures by penalizing violations of underlying differential equations within the objective function. For problems governed by a PDE such as the Helmholtz equation

$\nabla^2 P(r, s, f) + k^2 P(r, s, f) = 0,$

the residual computed from the network prediction is minimized at collocation points, enforcing the solution to obey the physics in a weak sense (Chen et al., 27 Jan 2026, Arora et al., 2023).

2. Permutation-Invariant Neural Network Constructions

A canonical permutation-invariant network structure leverages the deep-sets encoding, mapping a set of positions (or features) to a single latent, followed by permutation-invariant aggregation:

• Each element $x \in X$ is mapped via $\phi: \mathbb{R}^d \rightarrow \mathbb{R}^D$.
• The features are summed: $z = \sum_{x\in X} \phi(x)$.
• The aggregate $z$ is mapped to the output via $\rho: \mathbb{R}^D \rightarrow \mathbb{C}$ or $\mathbb{R}^k$.
• The function $f(X) := \rho(\sum_{x\in X} \phi(x))$ is permutation-invariant by construction.

For two-point interactions, as in region-to-region sound field reconstruction, $X = \{r, s\}$ (receiver and source positions). The PI-PINN outputs are symmetric: swapping $r$ and $s$ leaves $\phi(r) + \phi(s)$ invariant, hence $P(r, s, f) = P(s, r, f)$ is guaranteed, enforcing acoustic reciprocity (Chen et al., 27 Jan 2026).

For multi-object dynamics, pairwise permutation-equivariant layers are employed: $y_i = \frac{1}{N} \sum_{j=1}^N f(x_i, x_j).$ Other pooling operators (max, log-sum-exp) can be used to modulate the network’s inductive bias toward local or global interactions (Guttenberg et al., 2016).

3. Integration of Physical Constraints

Physics-informed loss terms are introduced by penalizing the residuals of the governing equation (ODE or PDE) at collocation points. For the Helmholtz equation, the residual is

$$R_i = \nabla^2 \hat{P}(r_i, s_i, f_i) + k^2 \hat{P}(r_i, s_i, f_i),$$

where $\hat{P}$ is the network prediction. The physics loss is

$$L_\mathrm{physics} = \frac{1}{N_\mathrm{PDE}} \sum_{i=1}^{N_\mathrm{PDE}} \| R_i \|^2,$$

computed at uniformly sampled collocation points in the spatial-frequency domain (Chen et al., 27 Jan 2026).

The total loss combines a data term (mean squared error between predicted and measured values) and the physics informed term, weighted by a hyperparameter $\lambda$: $$L_\mathrm{total} = L_\mathrm{data} + \lambda\,L_\mathrm{physics}.$$ For multi-particle systems modeled via permutation-invariant architectures, global physic constraints on conserved quantities (e.g., momentum, energy) can be included as explicit penalty terms in the loss (Guttenberg et al., 2016).

Explicit invariantization techniques for ODEs with Lie or finite symmetry groups include the Reynolds operator (group averaging) and reparameterizing state variables in invariant coordinates (e.g., elementary symmetric polynomials), yielding further training stability and lower error (Arora et al., 2023).

4. Training Recipes and Architectural Variants

Region-to-Region Sound Field Reconstruction

• The deep-set model for ATF prediction uses two separate two-layer MLPs (128 neurons per layer, tanh activations) for $\phi$ and $\rho$.
• Training is conducted for each frequency bin, separately for real and imaginary components.
• The dataset consists of measured room impulse responses (escape points on source and microphone grids).
• Adam optimizer is used with learning rate $10^{-3}$, $\beta_1=0.9$, $\beta_2=0.999$, for 50,000 steps or until loss plateaus.
• Ablation studies demonstrate the necessity of both permutation-invariance and physics-enforcing terms for robust generalization—removing either leads to loss of reciprocity or failure in physically unmeasured regions (Chen et al., 27 Jan 2026).

Multi-Agent and Particle Dynamics

• Permutational layers are stacked (typically 2–4), each being an MLP applied to all pairs or higher-order tuples.
• Skip connections (residual structure) enable the network to learn state increments over time.
• Choice of aggregation (sum vs. max) tailors the inductive bias toward global or local interactions, respectively.
• Explicit physical regularization (e.g., conserving energy or momentum) is achieved by adding global sum penalties (Guttenberg et al., 2016).

Symmetry-Based Invariant PINNs

• For systems admitting finite symmetries, group-averaging of the residual or modeling directly in explicit invariant coordinates ensures that the solution respects permutation symmetry at every iteration.
• Equivalent implementation can be achieved through symmetric weight tying in the network layers, $W = \alpha I + \beta11^T$, yielding permutation-equivariant transformations (Arora et al., 2023).

5. Empirical Performance and Comparative Evaluation

The region-to-region PI-PINN achieves substantial improvements over classical kernel methods (e.g., Kernel Ridge Regression) for sound field prediction:

• In anechoic and hemi-anechoic benchmarks, normalized mean squared error (NMSE) remains $-5$ to $-2$ dB for PI-PINN at frequencies above 1.1 kHz, where kernel baselines degrade, with gains of approximately $5$–$10$ dB.
• Field visualizations reveal that PI-PINN reconstructions faithfully capture spatial pressure distributions, while kernel baselines exhibit oversmoothing that erases critical spatial structure (Chen et al., 27 Jan 2026).

Ablation studies confirm that both the permutation-invariant encoder and the physics loss are essential. Removing either substantially increases prediction error, either by violating physical reciprocity or overfitting measured regions.

In multi-body particle dynamics, permutation-equivariant networks outperform dense MLPs of similar parameter count by more than a factor of $2\times$ in MSE, and demonstrate robust generalization to object counts outside the training set (Guttenberg et al., 2016).

Invariant PINNs based on Lie group or permutation symmetry display $1$–$4$ orders of magnitude improvement in training error, and lower sensitivity to discretization or optimizer instability, attributed to the "complexity-reducing" power of symmetry (Arora et al., 2023).

6. Broader Methodological and Theoretical Context

Permutation-invariant physics-informed architectures are closely related to the broader class of equivariant networks, which encode more general symmetry groups. In acoustic and dynamical systems, permutation invariance ensures physically meaningful behavior, such as reciprocity and indistinguishability.

Techniques such as invariantization via moving frames (for continuous symmetry) or Reynolds operators and symmetric polynomials (for finite/pure permutation symmetry) provide a mathematical foundation for integrating domain symmetries directly into the network’s architecture and loss, simplifying optimization landscapes and yielding improved empirical convergence properties (Arora et al., 2023).

Tables summarizing key empirical results and their implications:

Architecture	Application Domain	Symmetry Enforcement	Performance Benefit
PI-PINN (deep set + Helmholtz)	Sound field reconstruction	Explicit, via $\phi(r)+\phi(s)$	$5$–$10$ dB NMSE reduction versus KRR
Pairwise Perm-Equiv NN	Particle dynamics	Pairwise MLP + pooling	$2\times$ lower error vs. dense NN
Invariant PINN	ODEs with symmetries	Group averaging or invariants	$1$–$4$ orders of magnitude error reduction

7. Practical Considerations and Best Practices

• Select aggregation functions and permutation-invariant layer architectures (sum vs. max pooling) according to the physical interaction patterns—local vs. global.
• When explicit symmetry-breaking features exist (e.g., masses, radii), append them to per-object representations; otherwise, the permutation-invariant layer treats all inputs identically (Guttenberg et al., 2016).
• For PDE-based applications, ensure that network activation functions support differentiation to the required order for stable residual computation.
• Cross-validate loss weighting hyperparameters (e.g., $\lambda$ in $L_\mathrm{total}$) for optimal performance; empirical settings such as $\lambda=1$ have been validated in sound field applications (Chen et al., 27 Jan 2026).
• Evaluate on domain-relevant metrics, such as NMSE in dB for acoustic fields, or generalization across system sizes (object count) for dynamical systems.

Permutation-invariant physics-informed neural networks constitute a principled and empirically validated methodology for modeling physical systems with permutation symmetry, combining architectural and loss-level invariance with explicit physics regularization for improved fidelity and generalization (Chen et al., 27 Jan 2026, Guttenberg et al., 2016, Arora et al., 2023).