Graybox Modelling Strategy

• Graybox modelling strategy is a hybrid approach that blends physics-based models with machine learning to provide interpretability and predictive precision.
• It employs mixture-of-experts frameworks and alternating minimization to integrate expert parameters with gating weights, ensuring physical constraints are maintained.
• Empirical applications in system identification, control, and simulation demonstrate its efficiency in domains such as automotive dynamics, energy systems, and quantum calibration.

A graybox modelling strategy is a hybrid system identification and modelling paradigm that integrates physics-based priors (whitebox elements) with data-driven or machine learning representations (blackbox elements) through principled mathematical, optimization, and/or architectural frameworks. The aim is to achieve models that simultaneously offer interpretability, enforce physical constraints, and deliver predictive accuracy in regimes where neither purely first-principles nor purely data-driven approaches suffice. The approach has developed a rigorous foundation spanning mixture-of-experts modelling, composite surrogate-based optimisation, Bayesian and variational learning, and operator-theoretic formulations, with extensive application in engineering, control, and scientific disciplines.

1. Foundational Principles and Model Structure

The defining attribute of a graybox model is the explicit fusion of interpretable, physically-motivated local models (“experts”) with flexible, data-driven modules governed by a gating, surrogate, or coupling mechanism. In the canonical mixture-of-experts graybox framework, a sequence of regressors $x(t)\in\mathbb{R}^{n_x}$ and scalar outputs $y(t)\in\mathbb{R}$ is modelled as

$$\hat y(t) = \sum_{i=1}^M \omega_i(t)\; f_i(x(t);\theta_i)$$

where each expert $f_i$ is a parametric model (e.g., physics-based or low-order greybox ODE), $\theta_i$ are trainable parameters, and the weights $\Omega(t) = [\omega_1(t),\dots,\omega_M(t)]^\top$ reside in the simplex (nonnegative, sum to 1). The gating vector enables realtime switching or blending of experts according to the current regime (Leoni et al., 2024).

Physical constraints and first-principles knowledge are incorporated by selecting expert models $f_i$ that embed known physical structure (e.g., linear vehicle dynamics, reduced-order ODEs) and imposing convex or parameter constraints ($\theta_i\in\Theta_i$). This structure can extend to optimization contexts, where glassbox (analytic) and blackbox (non-analytic) subsystems are combined by introducing surrogate variables mapped to blackbox outputs in the optimization constraints (Hameed et al., 1 Sep 2025, Hameed et al., 24 Nov 2025).

2. Optimization and Learning Methodologies

Graybox model fitting often entails solving a joint or alternating minimization program for both expert parameters and gating weights, balancing global fit and local specialization. The objective function typically takes the form

$$J(\Theta, \Omega) = \sum_{t=1}^T \left[ \ell^{\mathrm{mix}} \bigl(y(t), \hat y(t) \bigr) + \beta \sum_{i=1}^M \omega_i(t)\, \ell_i^{\mathrm{loc}} \bigl(y(t), f_i(x(t);\theta_i)\bigr) \right] + \lambda_\theta \sum_{i=1}^M r_i(\theta_i) + \eta \sum_{t=2}^T \|\Omega(t)-\Omega(t-1)\|_2^2$$

where $\ell^{\mathrm{mix}}$ (ensemble loss), $\ell_i^{\mathrm{loc}}$ (expert-local loss), $r_i(\theta_i)$ (regularization), and a smoothness penalty ($\eta$) for the gating vector are jointly minimized under simplex constraints (Leoni et al., 2024).

Alternating minimization proceeds via block-coordinate descent: expert parameters are updated via (weighted) least squares or parallelized convex subproblems, and gating weights by quadratic programming. Coupled objectives can require augmented Lagrangian (ADMM) or stochastic approximation when mixing terms are strongly non-separable.

For optimization problems with both analytic and non-analytic components, trust-region and filter methods are employed. Surrogate models (linear, quadratic, GP-based) for blackbox functions are embedded in the subproblem, with trust-region radii and filter-based or funnel-based acceptance to control approximation error and ensure global convergence (Hameed et al., 1 Sep 2025, Hameed et al., 24 Nov 2025). Hessian information is projected or regularized to guarantee convexity and improve local convergence rates.

3. Interpretability and Physical Consistency

A central tenet of graybox modelling is the preservation of interpretability through (i) the explicit parametrization of experts in physically meaningful terms (e.g., mass, cornering stiffness, thermal conductance), and (ii) the gating mechanism which reveals regime allocations and switching policies. Regularization and smoothness penalties on the gating vector enforce realistic transitions and prevent overfitting to spurious data modes.

Physical constraint enforcement includes the selection of basis functions consistent with symmetry or conservation laws, as well as hard bounds or convex constraints on expert parameters (e.g., positivity of stiffness, consistency with kinematic bounds). In the optimization context, analytic derivatives for glassbox models are retained while blackbox modules are handled via local surrogates whose error is tightly controlled in the trust-region/funnel framework, guaranteeing model feasibility and well-posedness (Hameed et al., 24 Nov 2025).

4. Generalizations and Structural Variants

The graybox paradigm extends beyond mixture-of-experts. Notable variants include:

• Implicit graybox models in simulation: State variables are shared by both physics and DNN subsystems, requiring hybrid simulation engines capable of backpropagating through DNNs to supply Jacobian contributions during Newton–Raphson iterations. This allows for tight physical coupling and accurate sensitivity propagation, with modular training of DNN macromodels localized to active device boundaries (Agarwal et al., 2024).
• Surrogate-based Bayesian optimization: Graybox Bayesian optimization exploits functional decompositions (e.g., compartmental ODEs) by fitting surrogates (GPs) only to intractable submodules, propagating uncertainty through closed-form compositions. Acquisitions are designed to query only the most informative (sub)modules, reducing sample cost and accelerating convergence (Astudillo et al., 2022, Niu et al., 2024).
• Role in reinforcement learning and automata inference: Graybox methods leverage partial knowledge of system structure (e.g., state spaces, action supports, data guards) to accelerate value estimation, policy synthesis, and finite-state model learning. Dynamic taint analysis or interval MDPs reconstruct constraint structure, reducing query complexity and enabling learning in infinite or parametric state domains (Baier et al., 2023, Garhewal et al., 2020).

5. Practical Implementation Considerations

Implementation necessitates careful selection of expert model structure, surrogate fidelity, and hyperparameter tuning:

• Number of experts ($M$): Reflects the number of expected operating regimes. Must be selected a priori or via validation.
• Prior knowledge embedding: Maximizing interpretable, physics-based structure in all $f_i$ to minimize required data and enhance generalization.
• Surrogate/model selection: Fidelity and form (linear/quadratic/Taylor/GP) should reflect smoothness, nonlinearity, and data availability. Sliding-window and stochastic updates are advisable for large datasets.
• Hyperparameter selection ($\beta, \eta, \lambda_\theta$): Requires cross-validation due to non-convexity; smoothness penalties ($\eta$) should align with regime-switching expected in the application.
• Optimization stability: For non-convex objectives, multiple restarts are recommended; infeasibility restoration and step rejection criteria (e.g., funnel width in TR algorithms) must be carefully set (Hameed et al., 24 Nov 2025).
• Data sufficiency: Minimal number of commissioning points or standard component libraries aid model transfer to new systems (Sawant et al., 2019).

6. Empirical Performance and Applications

Graybox models consistently outperform both pure whitebox and pure blackbox approaches in empirical studies:

• System identification and control: In sideslip estimation for vehicles and quantum system identification, graybox mixtures achieve exact or near-exact recovery of expert weights and exhibit higher goodness-of-fit, with clear interpretability of regime selection (Leoni et al., 2024, Youssry et al., 2022).
• Industrial and process optimization: Trust-region graybox optimization yields order-of-magnitude reductions in iterations and black-box evaluations relative to traditional filter methods, with robust convergence across engineering benchmarks (Hameed et al., 1 Sep 2025, Hameed et al., 24 Nov 2025).
• Simulation: Hybrid graybox simulation reduces model state dimensions and runtime, delivers accurate Jacobians, and secures physically consistent behavior in large-scale power systems (Agarwal et al., 2024).
• Data-driven physical systems: In table-tennis trajectory prediction, the introduction of neural spin initializers and learned parameters synergistically enhances long-term predictive accuracy and supports real-time control (Achterhold et al., 2023).
• Broader application domains: Graybox approaches are usable in energy systems model predictive control, quantum device calibration, epidemiological model calibration, reinforcement learning, and automata learning (Sawant et al., 2019, Pathumsoot et al., 18 Aug 2025, Niu et al., 2024, Baier et al., 2023, Garhewal et al., 2020).

7. Limitations and Future Directions

Key limitations include the non-convexity of joint objectives, the need for cross-validation of multiple hyperparameters, and the requirement for sufficient data to reliably tune both interpretable and blackbox components. When functional relationships (e.g., gating weights) depend strongly on unobservable or unmeasured variables, graybox generalization may degrade, requiring the introduction of data-driven gating networks trained via supervised learning (Leoni et al., 2024). Overparameterization of blackbox elements erodes interpretability and may impair physical plausibility. For large-scale optimization and simulation, handling extrapolation and scaling DNN surrogates to unseen regimes remains a challenge (Agarwal et al., 2024). Model drift, especially in real-world noisy environments, motivates adaptive or transfer-learning extensions.

The graybox modelling strategy thus provides a rigorously-justified, extensible framework for scientific machine learning, embedding reliable domain knowledge and exploiting empirical flexibility to deliver superior accuracy, interpretability, and efficiency across a wide spectrum of applications (Leoni et al., 2024, Hameed et al., 1 Sep 2025, Hameed et al., 24 Nov 2025, Agarwal et al., 2024, Youssry et al., 2022, Niu et al., 2024, Sawant et al., 2019).