Multi-Scale SIREN-PINN Architecture

• The paper demonstrates that Multi-Scale SIREN-PINN uses dual SIREN branches with frequency-scaled sinusoidal activations to capture both macro- and micro-scale dynamics.
• It integrates physics-informed loss functions and coupled geometry-state learning to mitigate spectral bias and resolve high-frequency oscillations.
• The architecture achieves robust convergence and lower error metrics, outperforming conventional PINNs in applications like heat transfer and reaction-diffusion systems.

The Multi-Scale SIREN-PINN architecture is a neural PDE solver that combines sinusoidal representation networks and physics-informed loss functions to accurately model multiscale spatiotemporal dynamics, particularly on complex manifolds or in media with highly oscillatory coefficients. Distinct from conventional PINN frameworks employing smooth activations (e.g., Tanh, ReLU), Multi-Scale SIREN-PINN leverages frequency-diverse sinusoidal activations and network initialization to resolve both macro- and micro-scale physics, and introduces coupled geometry-state learning. It demonstrates utility in domains from multiscale heat transfer (Hintermüller et al., 2023) to stochastic reaction-diffusion in curved geometries (Chrisnanto et al., 13 Jan 2026).

1. Architectural Principles and Network Design

Multi-Scale SIREN-PINN uses dual fully-connected SIREN branches, each leveraging periodic activations $$h^{(l)} = \sin(\omega_0 W^{(l)} h^{(l-1)} + b^{(l)})$$ with frequency scaling factors $\omega_0$ specific to each layer and branch (Chrisnanto et al., 13 Jan 2026). The architecture typically comprises:

• Geometry Branch ($\Psi_\phi$): 5-layer SIREN mapping $x = (x, y)$ to curvature field $\kappa(x)$ with $\omega_0 = 30$ in the first layer, followed by lower-frequency layers.
• State Branch ($\Phi_\theta$): 8-layer SIREN mapping $(x, y, t)$ to complex system state $A(x, t) \in \mathbb{C}$, initialized at $\omega_0 = 10$ for the input layer.
• Initialization follows Sitzmann et al. (2020): weights $W^{(l)}$ and biases $b^{(l)}$ are drawn from $$\text{Uniform}\left(-\sqrt{6/n^{l-1}}/\omega_0, +\sqrt{6/n^{l-1}}/\omega_0\right)$$ and $\text{Uniform}(-\pi/\omega_0, +\pi/\omega_0)$ respectively.

This configuration is designed to mitigate spectral bias and enable representation of both low-frequency envelopes and high-frequency defect structures.

2. Multiscale Physical Modeling via PDE-Constrained Loss

The surrogate model approximates physical states and latent geometry. In the context of a curvature-perturbed Ginzburg-Landau equation, the physics residual for the PINN is

$$F_{\theta,\phi}(x, t) := \partial_t A_\theta - \mu A_\theta - (1 + ib) \nabla \cdot [D_0(1 + \alpha \kappa_\phi(x)) \nabla A_\theta] + (1 + ic) |A_\theta|^2 A_\theta$$

where $A_\theta(x, t)$ is the predicted amplitude field and $\kappa_\phi(x)$ is the learned curvature. Analogously, in heat-transfer settings, the fine-scale and coarse-scale problems are formulated as (Hintermüller et al., 2023):

• Fine-scale PDE: $A^\varepsilon u^\varepsilon = f^\varepsilon$ in $H$, with $$A^\varepsilon[u] = -\nabla \cdot (K^\varepsilon(x) \nabla u)$$ and $K^\varepsilon(x)$ rapidly oscillating.
• Coarse-scale PDE: Bilinear form $$b_{L[u]}(y, v) = \langle L[u] y, v \rangle_{Y^*, Y}$$, with $L[u]$ defining upscaled physics.

3. Loss Definitions and Multi-Objective Training

Multi-Scale SIREN-PINN employs a multi-objective loss integrating data fidelity, PDE residual, boundary conditions, spectral structure, and geometry regularization (Chrisnanto et al., 13 Jan 2026):

• Data loss: $L_{\text{data}}$ over sparse observations.
• PDE loss: $L_{\text{PDE}}$ on collocation points sampling the domain.
• Boundary loss: $L_{\text{BC}}$ enforces periodicity or Dirichlet constraints.
• Spectral loss: $L_{\text{Spec}}$ aligns predicted and empirical Fourier spectra.
• Curvature regularization: $L_{\text{Reg}}$ via total variation of $\kappa_\phi(x)$.

The total loss is:

$$L_{\text{total}} = \lambda_{\text{data}} L_{\text{data}} + \lambda_{\text{PDE}} L_{\text{PDE}} + \lambda_{\text{BC}} L_{\text{BC}} + \lambda_S L_{\text{Spec}} + \lambda_R L_{\text{Reg}}$$

Weights $\lambda_i$ are adaptively balanced using the gradient-norm ratio method, and coarse-scale regularization terms are used to accelerate convergence and act as preconditioners for low-frequency solution components (Hintermüller et al., 2023).

4. Training Protocols and Computational Workflow

Training begins with a warm-up phase (epochs $1$–$500$) where physics loss is omitted ($\lambda_{\text{PDE}} = 0$), followed by a coupled physics phase ($501$–$3000$ epochs) where all objectives are activated and balanced adaptively (Chrisnanto et al., 13 Jan 2026). Optimization is performed using Adam during exploration and L-BFGS for convergence refinement. Collocation strategies employ up to 100,000 spatiotemporal samples per epoch, with boundary and regularization points drawn randomly.

In hybrid PDE-constrained settings (Hintermüller et al., 2023), the workflow alternates between adjoint-based gradient computation (leveraging automatic differentiation for neural objectives and sparse matrix solves for finite element state and adjoint systems) and neural parameter updates. Periodic re-weighting of loss terms ensures stability through training.

5. Spectral Phase Transitions and Optimization Dynamics

A defining feature is the observed Spectral Phase Transition, typically occurring near epoch $\sim 2,100$ (Chrisnanto et al., 13 Jan 2026). Prior to transition, loss stagnates at $\sim 10^0$; during transition, cooperative minimization of physics and geometry losses drives $L_{\text{total}}$ down four orders of magnitude ($10^0 \rightarrow 10^{-5}$), signifying alignment of high-frequency modes and Pareto-optimal convergence. Beyond transition, minor oscillatory behavior is present as spectral alignment is refined.

In multiscale heat transfer, inclusion of coarse-scale regularization prevents overfitting to coarse gaps and improves the convergence rate and solution fidelity, as measured by residual norms and $L^2$ errors (Hintermüller et al., 2023).

6. Quantitative Performance and Ablation Results

Performance metrics include (Chrisnanto et al., 13 Jan 2026):

• Relative $L_2$ error: $$\varepsilon_{L_2} = \|A_\theta - A_{GT}\|_2 / \|A_{GT}\|_2$$.
• Defect-count error: $$\Delta N_{\text{defects}} = |N_{\text{pred}} - N_{\text{GT}}|$$.
• Curvature reconstruction: $\varepsilon_\kappa$, Pearson correlation $\rho$.

In chemical turbulence experiments,

Architecture	$\varepsilon_{L_2}$	$\Delta N_{\text{defects}}$	$\rho$
Multi-Scale SIREN-PINN	$1.92 \times 10^{-2}$	$< 1$	$0.965$
ReLU-PINN	$4.12 \times 10^{-1}$	$24.5$	—
Tanh-PINN	$1.56 \times 10^{-1}$	$12.2$	—
Fourier-PINN	$8.45 \times 10^{-2}$	$5.8$	—
Single-Scale SIREN-PINN	$9.15 \times 10^{-2}$	—	$0.82$

In multiscale heat transfer (Hintermüller et al., 2023), hybrid PINN methods lower $L^2$ errors to $O(10^{-2})$–$O(10^{-3})$ versus stalled convergence ($O(10^{-1})$–$O(10^{0})$) in non-hybrid baselines. The regularization term preconditions low-frequency solution components and leads to sharper, steadier convergence.

7. Applications and Broader Implications

Multi-Scale SIREN-PINN enables direct resolution of spatiotemporal chaos in reaction-diffusion systems, reconstruction of hidden curvature fields from wave dynamics, and accurate solution of ill-posed inverse pinning problems on latent manifolds (Chrisnanto et al., 13 Jan 2026). In heat transfer, it renders efficient upscaling in media with rapidly varying properties, leveraging hybrid neural/finite-element solvers (Hintermüller et al., 2023). The framework facilitates mesh-free, data-driven discovery of physical and geometric system properties, and supports engineering of passive control strategies via geometry-informed catalytic design.

This methodology suggests a paradigm shift for multiscale modeling, allowing for the synthesis of emergent physics and complex geometry, and improved solver convergence properties in the presence of multiple scales and high-frequency features.