Real-Time Iteration Schemes for NMPC

• RTI schemes are online optimization methods that perform a single SQP update per interval, enabling real-time NMPC with reduced computational demand.
• They leverage warm-started trajectories and local linearizations to efficiently solve discretized nonlinear problems on embedded systems.
• Algorithmic variants extend RTI to distributed, robust, and learning-driven control applications while ensuring precise execution-time guarantees.

A Real-Time Iteration (RTI) scheme is an online optimization method tailored for rapidly solving nonlinear optimal control and estimation problems, most notably nonlinear model predictive control (NMPC), in real time. RTI principles are fundamentally rooted in performing a single Sequential Quadratic Programming (SQP) update (or an analogous first-order step) per sampling interval, using warm-started state and input trajectories from the previous interval. This approach leverages temporal structure and problem sparsity to drastically reduce computational demand relative to full optimization, enabling closed-loop NMPC even on resource-constrained embedded platforms and in time-sensitive scenarios. RTI schemes have further inspired algorithmic innovations in distributed and robust MPC, diffusion model inference, and reinforcement learning architectures.

1. Fundamental Structure and Mathematical Framework

The RTI paradigm begins from a general finite-horizon nonlinear optimal control problem (OCP): $$\begin{aligned} &\min_{x(\cdot),u(\cdot)}\, J[x(\cdot),u(\cdot)] \ &\text{s.t.}\ \dot{x}(t)=f(x(t),u(t)),\ x(0)=x_0 \ &x(t)\in\mathcal{X},\ u(t)\in\mathcal{U},\ G(x(t),u(t))\leq0\ . \end{aligned}$$ For real-time computation, the continuous problem is discretized (preferably by methods that preserve stability, e.g., implicit Euler for stiff dynamics (Wagner et al., 2024)). At each control step, the nonlinear program is approximated locally (via Taylor expansions and Gauss–Newton Hessian) about a shifted “guess” trajectory derived from the previous solution. This yields a quadratic program (QP) for state and input increments, which is solved exactly once per step, rather than to full convergence (Wagner et al., 2024, Carlos et al., 2020, Zanon et al., 2020).

Central algorithmic steps:

1. Warm start: Shift previous optimal trajectories one step ahead to initialize the nominal.
2. Linearization and discretization: Compute local linearizations of dynamics and constraints about the warm-start guess.
3. Quadratic Programming: Formulate and solve the condensed or full QP for state and input increments.
4. Feedback update and control application: Update trajectories and apply the first (or an increment) of the optimized input.
5. Advance: Measure new state, shift the horizon, and repeat.

Formally, the “feedback” update at each step implements a Newton-type correction to the guess, yielding local contraction toward the exact solution under standard regularity and smoothness assumptions (e.g., LICQ, twice differentiable dynamics and cost, strong second-order sufficiency) (Wagner et al., 2024, Zanon et al., 2020, Frey et al., 2024).

2. Algorithmic Variants and Extensions

RTI admits numerous variants and specialization, often driven by task or hardware constraints.

• Advanced-Step RTI (AS-RTI) and Multi-Level Iteration (MLI): To increase accuracy without sacrificing latency, additional “advanced-step” (pre-feedback) Newton- or quasi-Newton iterations can be performed using a predicted state. MLI structures allow trade-offs between computational cost and contraction rate, as formalized in a four-level hierarchy: Level D (full SQP), C (fixed Jacobian), B (zero-order), and A (single-predictor) (Frey et al., 2024).
• Partial condensing and block-structured QPs: The quadratic subproblem can be generated in sparse, dense, or partially condensed forms, balancing memory use and solve speed. Partial condensing groups multiple state-control pairs, reducing the QP’s dimension while preserving block structure, and exploits Riccati or other structured recursions for efficient solutions (Carlos et al., 2020, Wu et al., 2024).
• Sampling-augmented, adaptive or robust RTI: For applications with non-convexity or sudden constraint changes (e.g., adaptive friction or robust tube NMPC), warm-starts can be further enhanced via a library of sampled, dynamically feasible trajectories, with the candidate best suited (in terms of cost or feasibility) used for RTI initialization (Svensson et al., 2019, Frey et al., 2023).
• Distributed and decentralized RTI: In multi-agent or spatially distributed systems, a decentralized RTI scheme solves a bi-level decentralized SQP by applying one outer iteration per sampling instant, with primal and dual variables coupled via local consensus constraints and only peer-to-peer communication required for neighbor averaging (using ADMM) (Stomberg et al., 2024).
• First-order and alternative RTI approaches: Pure first-order methods—such as single-step projected-gradient or feedback-linearized PI algorithms—can be used to implement RTI-like control for quadratic programs (QP) arising in linear MPC, with theoretical guarantees on equilibrium and contraction rate (Pirrera et al., 6 Sep 2025).

3. Convergence, Performance, and Theoretical Guarantees

Classical RTI, under the assumptions of smoothness and regularity (e.g., LICQ, strict complementarity, Lipschitz continuity), ensures quadratic contraction of the primal-dual solution toward the true KKT point of the current OCP. The resulting closed-loop system inherits practical stability: the RTI controller tracks the optimal cost and constraint satisfaction up to an error of $O(\|\Delta x_0\|^2)$ (where $\Delta x_0$ is the state prediction-update residual) (Carlos et al., 2020, Zanon et al., 2020, Frey et al., 2024, Wagner et al., 2024).

For distributed variants, the combination of decentralized SQP convergence results (q-linear rate with a fixed number of ADMM inner steps per outer iteration) and standard RTI stability theorems ensures local exponential stability of the coupled system-optimizer dynamics (Stomberg et al., 2024).

Advanced-step and multi-level RTI methods admit error bounds explicating the effect of predictor-state inaccuracy and the number of inner iterations on contraction and feasibility. Explicit computational complexity bounds are provided for various algorithms, ranging from $O(N(n_x+n_u))$ per QP solve for banded, sparse structured solvers to $O(N^2)$ for single-step first-order methods in linear settings (Wagner et al., 2024, Carlos et al., 2020, Pirrera et al., 6 Sep 2025, Wu et al., 2024).

4. Implementation and Computational Complexity

RTI schemes are designed for real-time feasibility, even in embedded and resource-limited contexts:

• Sparse QP and Riccati solvers: Block-banded structure and Riccati-based factorizations ensure $O(N(n_x+n_u))$ to $O(N n_x^3)$ complexity per Newton or SQP step, far less than the $O(N^3)$ of fully dense approaches (Wu et al., 2024, Carlos et al., 2020, Carlos et al., 2020, Frey et al., 2023).
• Hardware-optimized libraries: Packages such as HPIPM and BLASFEO offer hand-tuned routines for block solves and small dense factorizations, exploiting vectorization and cache structure (Carlos et al., 2020, Frey et al., 2023).
• Preparation and feedback phase split: Costly model linearizations and condensed matrix generation are performed in a “preparation” phase, potentially before state measurement arrives, minimizing feedback latency (Frey et al., 2024, Frey et al., 2023).
• Execution-time certification: Algorithms have been developed that admit exact, data-independent flop counts and thus hard real-time execution-time guarantees, critical for certifiable embedded control (Wu et al., 2024).
• Empirical timings: RTI QP solve times of $7$–$15$ ms (typical) for horizons up to $N=50$ in autonomous vehicles, quadrotors, and robotic platforms; sub-millisecond timing is observable in linearized or simpler setups (Wagner et al., 2024, Carlos et al., 2020, Pirrera et al., 6 Sep 2025, Frey et al., 2023).
• Integration with robust and stochastic outer loops: RTI meshes with robust/shaped MPC loops (e.g., zoRO) that propagate uncertainties outside the core QP, resulting in cycle times an order of magnitude faster than previous methods (Frey et al., 2023).

5. Application Domains

RTI schemes underpin numerous practical optimization-based control and inference pipelines across domains:

• Autonomous vehicle trajectory tracking: RTI with implicit discretization and robust constraint handling enables real-time NMPC even for stiff vehicle models, with tracking error comparable to full NLP solutions but computation time reduced by a factor of 7 (Wagner et al., 2024).
• Quadrotor and underactuated systems: Real-time NMPC with RTI, leveraging partial condensing and hardware-tuned QP solvers, achieves single-digit-millisecond cycle times and robust closed-loop performance in agile flight with communication delay compensation (Carlos et al., 2020).
• Nonconvex trajectory planning under changing constraints: Sampling-augmented warmstarts for RTI yield improved collision avoidance and constraint satisfaction, overcoming local minima and sensitivity to friction changes (Svensson et al., 2019).
• Distributed, multi-agent, or spatially coupled systems: Decentralized RTI with distributed optimization and only peer-to-peer communication accomplishes coordinated NMPC with provable stability and subsecond solve times per subsystem (Stomberg et al., 2024).
• Diffusion-model-based robotic action selection: Adaptations of RTI support low-latency streaming inference for diffusion policies, by replacing full denoising with a minimal diffusion schedule per “chunk” and warm-starting from prior outputs (Duan et al., 7 Aug 2025).
• Reinforcement learning with embedded NMPC: RTI permits embedding NMPC as an efficient, single-step value estimator inside RL loops, supporting backpropagation of policy gradients through RTI’s QP solver (Zanon et al., 2020).
• Wave energy conversion: First-order, single-iteration RTI schemes achieve order-of-magnitude reductions in sampling time, substantially improving control fidelity and harvested energy under identical hardware (Pirrera et al., 6 Sep 2025).
• Chance-constrained and robust tube MPC: RTI seamlessly integrates with zero-order robust optimization outer loops, delivering efficient tube-MPC with full real-time feasibility on embedded CPUs (Frey et al., 2023).

6. Limitations, Design Considerations, and Best Practices

RTI’s primary compromise is between immediacy—performing only one corrector step—and proximity to the true NLP optimum. This yields only local contraction, and states that are far from the previous trajectory or radically changing environments can result in suboptimality or constraint violations. Robustness can be enhanced through advanced-step strategies, sampling-augmented warmstarts, adaptive constraint updating, and additional post-solve feasibility projections (Frey et al., 2024, Svensson et al., 2019). For hard or safety-critical applications, execution-time certification and tight flop-counting via factorized Riccati ensure deadline adherence (Wu et al., 2024).

Best practices include: judicious selection of SQP level and number of pre-update iterations (favoring higher levels for near-optimality or in the presence of stiff dynamics), tailoring predictor step use to anticipated state evolution, exploiting warmstarts and trajectory shifting, and tuning condensing strategy to available memory and CPU. For distributed or large-scale problems, local consensus enforcement and leveraging sparsity in subsystem interactions are critical (Stomberg et al., 2024).

7. Research Frontiers and Emerging Directions

Current RTI-related research is advancing several axes:

• Algorithmic generalization: Extension from classical Gauss-Newton approaches to first-order, projection-based controllers, enabling even lower computational cost for convex MPC problems (Pirrera et al., 6 Sep 2025).
• Robustness and uncertainty management: Integration with outer-loop robust optimization and stochastic uncertainty propagation, making RTI suitable for tube-based and chance-constrained formulations (Frey et al., 2023).
• Distributed and hierarchical implementation: Scaling RTI to multi-agent, multi-level, and networked systems with only neighbor-to-neighbor communication, and leveraging block-sparse solvers (Stomberg et al., 2024).
• Execution-time certification: Precise, dimension-dependent flop-counting and O(N) factorized Riccati recursions provide certifiable timing for embedded deployment (Wu et al., 2024).
• Real-time inference in generative and sequential modeling: RTI-inspired warm-start approaches are finding new roles in reducing inference time for large diffusion-model policies in robotics, with theoretical contractivity conditions for accelerated convergence (Duan et al., 7 Aug 2025).
• Integration with reinforcement learning: Differentiable RTI solvers allow tight and efficient coupling with RL trust-region and value-based algorithms, supporting closed-loop learning and policy adaptation (Zanon et al., 2020).

A plausible implication is that RTI will continue to serve as a bridging paradigm between high-fidelity nonlinear optimization and real-time implementability in embedded and learning-driven systems. Emerging algorithmic advances are enlarging its range of applicability, incorporating distributed, robust, and data-driven extensions.