Adaptive and Learning-Based RMPC

• Adaptive and learning-based RMPC is a control framework that integrates machine learning techniques with robust MPC to dynamically adjust control actions while reducing conservatism.
• It employs methods like uncertainty set learning, tube adaptation, and meta-learning to maintain safety, recursive feasibility, and improved performance despite model mismatches.
• This approach has demonstrated practical gains including reduced tracking error and computation time across diverse applications such as autonomous driving, microgrid control, and robotics.

Adaptive and learning-based Robust Model Predictive Control (RMPC) integrates online or offline learning mechanisms—such as machine learning, meta-learning, or reinforcement learning (RL)—with robust MPC frameworks to mitigate model mismatch, enable adaptation to time-varying environments, and reduce conservatism in constraint satisfaction. This paradigm aims to maintain the safety and recursive feasibility guarantees of robust MPC while leveraging data-driven adaptation to capture uncertainties, reduce conservatism, and improve control performance across heterogeneous or nonstationary operating regimes.

1. Core Principles and Motivations

Conventional robust MPC enforces constraint satisfaction and recursive feasibility by designing tightened constraint sets or invariant tubes with respect to worst-case uncertainty bounds. This conservative approach ensures robust safety but often yields suboptimal performance, particularly when model uncertainties are overestimated or when the system operates in environments with large, time-varying, or partially-known disturbances. The adaptive and learning-based RMPC framework augments traditional robustification with online identification, learning-based adaptation, and data-driven parameter and policy optimization, enabling:

• Sample-efficient adaptation: Rapid policy adjustment to changing system dynamics or constraint structures (Yang et al., 2024, Zhang et al., 2023).
• Reduced conservatism: Shrinking of tubes or tightening margins when learned confidence in process models increases (Kiani et al., 2023, Petrenz et al., 15 Apr 2025).
• Improved generalization: Transferability and task-agnostic performance across classes of systems or tasks, through meta-learning or Bayesian priors (Yang et al., 2024, Sinha et al., 2022).
• Safety with performance: Maintenance of robust recursive feasibility while leveraging learning for less conservative and more performant control (Geurts et al., 28 May 2025, Li et al., 2023).

2. Algorithmic Structures in Adaptive RMPC

2.1 Learning-augmented Tube-based RMPC

Tube-based RMPC, such as the classic Mayne formulation, separates the closed-loop trajectory into a nominal center and an error bound (tube), applying a feedback policy to keep the error within a robust positive invariant (RPI) set. Learning augments this procedure by:

• Uncertainty set learning: Online estimation of disturbance or parameter bounds, either by set-membership identification (Aboudonia et al., 2024, Petrenz et al., 15 Apr 2025), Gaussian process regression (Kiani et al., 2023), or Bayesian update of parameter confidence sets (Sinha et al., 2022).
• Tube/adaptive tightening: At each time, the invariant tube and associated state/input constraints are recomputed based on the learned uncertainty. For instance, GP posterior variance is used to adapt tube cross-sections in microgrid voltage control (Kiani et al., 2023), while set-membership approaches shrink parameter uncertainty for coupled interconnected systems (Aboudonia et al., 2024).
• Hybridization with RL policies: Tube RMPC is sometimes integrated as a safety filter that projects RL-generated actions onto the subset certified safe by robust invariance (Lu et al., 2024, Li et al., 2023).

2.2 Meta-learning and RL-based RMPC

Learning-based RMPC leverages RL and meta-learning to achieve adaptation and generalization:

• Meta-RL optimizers: Sampling-based MPC controllers (e.g., MPPI) are optimized via meta-RL to learn task-adaptive update rules for controller parameters, using a bi-level gradient-based meta-optimization (Yang et al., 2024). Such optimizers enable fast, few-shot adaptation to new tasks by encoding shared structure across task families.
• Recurrent policy parameterization: Recurrent neural networks trained by backpropagation through time can learn explicit approximators of traditional MPC policies, enabling adaptive horizon selection and fast online synthesis (Liu et al., 2021, Liu et al., 2021).
• RL-driven horizon and tightening adaptation: RL agents can learn to select prediction horizons (Bøhn et al., 2021), constraint-tightening variables, or even MPC cost weights, trading off computation against performance and robustness.

2.3 Uncertainty Cancellation and Bayesian Priors

The adaptive robust "estimate-and-cancel" paradigm projects the learned uncertainty onto the control input domain, actively cancelling identifiable components and reducing the size of the robust disturbance set. Bayesian meta-learning may be used to calibrate priors and feature maps for more informative and quickly converging online identification (Sinha et al., 2022).

3. Representative Methodologies and Empirical Results

Class	Key Features / Techniques	Representative Papers
Learning-augmented Tube RMPC	GP or SMI-adaptive tubes, online set-tightening, recursive feasibility preservation	(Kiani et al., 2023, Aboudonia et al., 2024)
Meta-RL-optimized MPC	MAML-style meta-RL, few-shot optimizer adaptation, task distribution generalization	(Yang et al., 2024)
RL-driven Adaptive Schemes	RL for horizon/tightening/parameter tuning, safe RL with tube filters, RL based ellipsoid-tube adaptation	(Bøhn et al., 2021, Lu et al., 2024, Esfahani et al., 2021, Hori et al., 18 Dec 2025)
Explicit Recurrent Policy RMPC	Recurrent neural network policy, offline Bellman-decomposition training, adaptive horizon selection at runtime	(Liu et al., 2021, Liu et al., 2021)
Iterative/Set-Membership RMPC	Iterative terminal cost/set learning, shrinking uncertainty sets via data, adaptation over episodes	(Petrenz et al., 15 Apr 2025)
Estimate-and-cancel (ARMPC)	Nonlinear model uncertainty structure, online feature-based estimation, robust input set tightening with active cancellation	(Sinha et al., 2022)
Safe RL-RMPC Hybridization	RL/MPC integration for certified safety, RL for long-horizon objectives, tube-RMPC as safety filter	(Lu et al., 2024, Li et al., 2023, Zhang et al., 2023)
Classifier-Driven Adaptive MPC	BO/classifier-driven hyperparameter adaptation (e.g., temperature, control noise), online retraining and selection during MPC operation	(Guzman et al., 2022)
Contingency/LB-MPC Fusion	Multi-horizon contingency planning: robust RMPC horizon for safety, learning-based MPC horizon for performance, tight theoretical safety guarantees	(Geurts et al., 28 May 2025)

Empirical gains across these works include significant reductions in tracking error, computation time, and closed-loop cost compared to classical or non-adaptive RMPC, with preservation or improvement of safety and constraint satisfaction. For example, meta-RL optimizers achieve a ≈30% reduction in tracking error after 5 adaptation steps compared to non-meta RL optimizers (Yang et al., 2024), and GP-adaptive tube-RMPC in microgrids reduces voltage THD below both fixed-tube RMPC and nominal MPC, adapting tubes in real time based on the learned disturbance (Kiani et al., 2023). Adaptive regression-based MPC achieves a 35–65% reduction in QP solve time without significant loss in performance (Mostafa et al., 2022).

4. Theoretical Guarantees and Safety

Central to adaptive, learning-based RMPC is the rigorous maintenance of recursive feasibility, constraint satisfaction, and (robust) stability under adaptive or learned uncertainty sets. Key guarantees include:

• Recursive Feasibility: Ensured either by tube updates that are non-increasing with high probability (e.g., when using set-membership identification (Aboudonia et al., 2024), Bayesian confidence sets (Sinha et al., 2022), or when the learning horizon constraints do not become tighter than the original robust MPC tubes (Geurts et al., 28 May 2025)).
• Input-to-State Stability: Tube-based RMPC, when equipped with RPI tubes, ISS Lyapunov functions, or contraction conditions, retains closed-loop input-to-state stability despite online adaptation (Kiani et al., 2023, Aboudonia et al., 2024).
• Safety under Learning: Safe RL-RMPC couplings enforce that all RL policies are certified by a robust tube filter for constraint satisfaction at every time-step—even during early, unsafe exploration phases in RL training (Lu et al., 2024, Li et al., 2023).
• Meta-learning adaptation bounds: Meta-optimized optimizers for MPC yield few-shot adaptation to out-of-distribution tasks with preserved constraint satisfaction, as the meta-objective is explicitly constructed to maximize rapid expected return on new tasks (Yang et al., 2024).
• Statistical confidence preservation: Bayesian or set-membership identification approaches ensure the true parameter remains within the confidence or uncertainty set with specified probability (e.g., P{parameter in set for all t } ≥ 1−δ), and the corresponding RMPC constraint tightening decays monotonically (Sinha et al., 2022, Petrenz et al., 15 Apr 2025).

5. Application Domains and Practical Impacts

Adaptive, learning-based RMPC has demonstrated efficacy across a range of domains, notably:

• Autonomous and Mixed Traffic Driving: Online adaptation via recurrent RL and tube tightening for urban driving with time-varying vehicle parameters, showing robust constraint satisfaction across road friction, tire, and mass changes (Zhang et al., 2023). Hybrid learning-based RMPC achieves a 10.88% increase in energy efficiency compared to standard RMPC while guaranteeing zero collisions during RL agent exploration (Lu et al., 2024).
• Microgrid and Power Systems: GP-based tube adaptation in voltage control tasks enables low conservatism and robust operation in the presence of unpredictable harmonic loads and device variations (Kiani et al., 2023).
• Large-scale Interconnected Systems: Set-membership adaptive RMPC partitions adaption and control phases across subsystems, maintaining recursive feasibility and input-to-state stability with decentralized computation and communication (Aboudonia et al., 2024).
• Robotics and Trajectory Tracking: Approximate robust NMPC with RL-adapted ellipsoid tubes enables real-time, robust navigation and tracking (e.g., wheeled robots), improving closed-loop cost by 8–25% relative to non-adaptive RMPC (Esfahani et al., 2021).
• Iterative and Episodic Tasks: Learning-based terminal set and cost adaptation yields reduced required horizon and computation time in repetitive tasks under slowly-varying parametric uncertainties (Petrenz et al., 15 Apr 2025).

6. Open Challenges and Future Directions

Despite substantial progress, several important research challenges remain:

• Rigorous Stability under Deep Neural Policy Learning: While empirical recursive feasibility is preserved in many RL-RMPC integrations, theoretical guarantees—especially when parameterizing policies with deep neural networks—are underdeveloped.
• Scalability and Real-time Solution: While offline-trained recurrent policies (Liu et al., 2021) and regressor-based horizon selection (Mostafa et al., 2022) have reduced computation times drastically, the scalability of learning-based RMPC under distributed, high-dimensional, or hybrid discrete-continuous settings is an active area.
• Adversarial and Distributional Robustness: Extension of adaptive RMPC to adversarial (distributionally robust) settings, where the uncertainty is learned under worst-case or risk-averse scenarios, is progressing with reach-avoid policy iteration and adversarial RL architectures (Li et al., 2023, Schuurmans et al., 2020).
• Safe Exploration in RL-RMPC Hybrids: Best mechanisms to balance RL exploration and robust constraint satisfaction, especially during early training ('exploration under invariance filters'), remain open.
• Domain Generalization and Meta-learning: How best to meta-train optimizers or priors for deployment in completely novel or highly non-stationary domains, especially with limited adaptation data, is a subject of active investigation (Yang et al., 2024, Sinha et al., 2022).

The integration of learning-based adaptation with the formal guarantees of robust MPC continues to advance the reliability and performance of control algorithms in uncertain, data-rich, and highly dynamic environments. The spectrum of paradigms—from meta-learned optimization engines, through online adaptive tube tightening, safe RL-MPC coupling, to iterative learning of terminal ingredients—reflects a diverse and expanding field with both deep theoretical and significant practical impact.