Unified Operator Network Framework

• Unified operator networks are neural architectures that learn families of operators by mapping input functions to outputs using shared and task-specific components.
• They employ a shared encoder and per-operator decoders to enable scalable multi-task learning while reducing parameter growth and computational costs.
• Supported by universal approximation theory, these networks demonstrate improved empirical performance and data efficiency in solving parametric PDEs and multi-task problems.

A unified operator network is a neural or symbolic architecture designed to approximate, represent, or learn a family of operators—typically those mapping input functions or parameters to output functions—in a single, flexible model. This paradigm generalizes classical single-operator neural operators by constructing a single network, or a shared set of sub-networks, that can efficiently and accurately address multiple operator learning problems simultaneously or handle a parametric continuum of operators. Unified operator networks have foundational roles in scientific machine learning for parametric PDEs, symbolic regression, and multi-tenant or multi-task systems, offering theoretical and practical advantages in data efficiency, parameter sharing, and scalability.

1. Mathematical Foundations and Core Architectures

Central to the unified operator network framework is the extension from single-operator learning (SOL) to multi-operator learning (MOL), treating the operator as either a finite collection of maps $\mathcal{G}_j: \mathcal{U}\to \mathcal{V}_j$ for $j=1,\ldots,M$ or as a parametric family $$\{G[\alpha]: \mathcal{U}\to \mathcal{V}\}, \alpha\in\mathcal{W}$$ (Zhang, 2024, Weihs et al., 29 Oct 2025). Key architectures exhibit the following structure:

• Shared Encoder / Branch Network: A neural mapping $E(u;\theta_E)$ encodes discretized input $u$ into a low-dimensional representation, shared across all operators.
• Per-Operator Decoders / Trunk Networks: For each operator $j$, a distinct decoder $b^j_k(x; \theta_{b^j})$ (or, in the parametric regime, a parametric branch/trunk) generates output basis functions specialized to $j$ or parameter $\alpha$.
• Unified Expansion: The network output at any point $x$ is given by a bilinear or multilinear expansion,

$$\hat y^j(x) = \sum_{k=1}^K b^j_k\bigl(x; \theta_{b^j}\bigr)\, c_k\bigl(E(u; \theta_E)\bigr),$$

or, for parametrized operators,

$$\hat y(x) = \sum_{k=1}^N \sum_{i=1}^M \tau_k(x)\, b_{k,i}(u)\, L_{k,i}(\alpha),$$

with $\tau_k(x)$, $b_{k,i}(u)$, and $L_{k,i}(\alpha)$ being coordinate, input, and parameter basis functions, respectively (Weihs et al., 29 Oct 2025).

This approach generalizes classical DeepONet, allows for plug-in extensions (e.g., BelNet, MIONet), and accommodates both MOL (finite $M$) and continuum-of-operators regimes.

2. Distributed and Modular Training Methodologies

Unified operator networks leverage distributed, modular training algorithms to scale MOL without proportional growth in parameters or computational burden. The MODNO algorithm (Zhang, 2024) exemplifies this strategy:

1. Initialization: Shared encoder weights ($\theta_E$) and operator-specific decoder weights ($\theta_{b^j}$) are initialized.
2. Alternating Steps:
   • Each trunk network $\theta_{b^j}$ is updated independently on its operator-specific data, holding $\theta_E$ fixed.
   • The shared encoder $\theta_E$ is updated using the aggregate gradients from all operators, thus centralizing learning of input representations.
3. Communication: Only encoder weights and gradients need central coordination; trunk updates are fully decoupled.
4. Cost Control: Subsampling (using a fraction $q$ of the pooled data for encoder updates) further reduces global cost, yet preserves accuracy for reasonable $q$.

This training protocol enables training of a single network for $M$ operators at an average cost comparable to independent SOL—without requiring a decoder nets’ parameter count to scale linearly in $M$. Operators with limited data benefit from representation sharing with data-rich peers.

3. Theoretical Guarantees: Universal Approximation and Scaling Laws

Unified operator networks possess rigorous universal approximation properties:

• Continuous Regime: If both parameter-to-operator ($\alpha\to G[\alpha]$) and input-to-output ($u\to G[\alpha][u]$) mappings are continuous, then for any precision $\epsilon$, a unified network of MONet/MNO (or MODNO) type achieves sup-norm error $\leq\epsilon$ uniformly in $(\alpha, u, x)$ (Weihs et al., 29 Oct 2025).
• Measurable/L2 Regime: For Borel-measurable, integrable operator families, unified operator networks approximate to any $L_2$ error threshold.
• Lipschitz Regime (Scaling Laws): For families with Lipschitz smoothness in parametric index $\alpha$ and input $u$, explicit parameter count vs. accuracy scaling laws are established:

$$\epsilon \approx \left( \frac{\log\log N_\#}{\log\log\log N_\#} \right)^{-1/d_W},$$

where $N_\#$ is parameter count and $d_W$ is parametric dimension. This scaling is nearly double-exponential in the inverse accuracy, setting theoretical limits on practical network sizes (Weihs et al., 29 Oct 2025).

These results subsume and generalize the single-operator DeepONet/FNO scaling behaviors, and provide complexity guidelines for practical MOL deployments.

4. Empirical Performance and Data Efficiency

Extensive empirical benchmarking across multiple PDE families demonstrates the pragmatic advantages of unified operator networks:

Model / Task	Relative L2 Error (%)	Cost Efficiency
Single-DON (per-op baseline)	0.98–24.03	Linear in M
MODNO (MOL, q=1.0)	0.22–7.62	≈ single-DON
MNO-Large (continuum param.)	2.50–4.41	Sub-linear in task#

• On five MOL benchmarks, MODNO matches or surpasses per-operator training in 11/16 cases, without exceeding total cost. For operators with small datasets, MODNO often yields notably improved performance—demonstrating transfer via encoder sharing (Zhang, 2024).
• On parametric PDEs, MNO outperforms DeepONet, MIONet, and MONet, especially with increased capacity (scale from $1.2$M to $16.7$M parameters) (Weihs et al., 29 Oct 2025).
• Subsampling in encoder training (q down to 0.7) retains accuracies within 10% of baseline, offering practical cost control.

These results underscore unified operator networks’ utility in both sample efficiency and training scalability, with no compromise in accuracy and, for low-data operators, a distinct benefit.

5. Structural and Representational Extensions

The unified operator framework is highly modular and extensible:

• Plug-in Architectures: Any Chen–Chen–Lu type neural operator (DeepONet, BelNet, MIONet) can serve as the core network; the unification holds for architectures with shared encoders and per-task decoders (Zhang, 2024, Zhang et al., 2023).
• Operator Discretization Invariance: Discretization-invariant neural operators (e.g., BelNet) support operator learning where sensor locations differ per sample, and bases for input/output function space are learned, extending unified operator networks to mesh-invariant and heterogeneous data regimes (Zhang et al., 2023).
• Parallel Single-Operator Learning: A single network can learn a large collection of independent operators (multi-single-operator learning) by judicious parameter sharing. Remark 4.9 in (Weihs et al., 29 Oct 2025) shows that at most three components (param-approx, branch-approx, trunk-approx) suffice, with universal approximation retained.
• Limitations: Current unified operator networks do not automatically extrapolate to entirely unseen operators (operator forms absent from the training set). Extensions to physics-informed losses and improved data/modeling for discretization- and PDE-invariant learning are open directions.

6. Practical Implementation Considerations

Unified operator networks accommodate practical and scalable deployment:

• Parameter Budget: Total network parameters grow as $\dim(\theta_E) + \sum_{j=1}^M \dim(\theta_{b^j})$, sublinear compared to training independent networks when the encoder dominates.
• Distributed Training Efficiency: Per-operator training steps may be parallelized; only small, centralized steps for the encoder are needed.
• Subsampling Strategies: Fare well empirically, especially for well-represented operator classes.
• Applicability: Unified operator networks extend naturally to operator families arising in physical sciences (parametric and multi-physics PDEs), controlled dynamical systems, and heterogeneous sensor networks.
• Extension Points: Incorporation of discretization-invariance, orthogonality/stability regularization, joint learning of operator structure (architecture search), and multi-fidelity/physics-guided learning.

7. Connections, Generalizations, and Outlook

Unified operator networks draw on, and generalize, several foundational lines of research:

• DeepONet and its Derivatives: The division into input and output branches, with a linear or bilinear expansion, is core to most unified operator architectures.
• Multitask and Multiphysics Learning: The framework directly supports learning multiple tasks/physics in one model through shared representations and per-task decoders.
• Symbolic and Neural Operator Synthesis: Techniques from symbolic regression and neural operator learning can be unified under architectures that handle both symbolic and numerical operator families (Li et al., 9 May 2025).
• Parametric Neural Operators: The abstraction to operator-index parameter spaces allows both discrete and continuous task distributions (Weihs et al., 29 Oct 2025).
• Robustness and Data Fusion: Discretization- and mesh-invariant architectures extend unified operator training to non-uniform, data-fusion scenarios (Zhang et al., 2023).

The unified operator network paradigm centralizes the theoretical landscape and offers scalable, efficient solutions for large-scale scientific machine learning and multi-task operator approximation. Its future trajectory will likely involve deeper integration with scientific prior knowledge, improved universality across operator classes, and further advances in efficient distributed and parallel training.