Online Residual Learning Paradigm

• Online Residual Learning is a hybrid paradigm that combines interpretable, physics-based or offline models with an online adaptive residual corrector to address prediction discrepancies.
• It employs lightweight neural networks or parametric functions for real-time updates, ensuring robust, sample-efficient adaptation under dynamic and uncertain conditions.
• The approach is applied in robotics, MPC, sim-to-real adaptation, and large-scale mapping, effectively balancing safety, stability, and computational efficiency.

Online residual learning is a hybrid, adaptive modeling and control paradigm in which a baseline model—often physics-based or derived from offline learning—provides coarse predictions or actions, while a supplemental residual module is trained and updated online to correct persistent or transient discrepancies between model outputs and empirical observations. This approach enables controllers and predictors to maintain stability and interpretability while adapting rapidly to novel dynamics, disturbances, or environment changes through real-time correction mechanisms. Online residual learning is now established in robotics, control, vision, sim-to-real adaptation, large-scale mapping, and sequential prediction. The paradigm capitalizes on the data efficiency and safety of strong prior models, leveraging additive neural corrections, context encoding, and global or probabilistic residual policies for improved robustness, generalization, and sample-efficient adaptation.

1. Fundamental Principles of Online Residual Learning

The defining structure of online residual learning involves the decomposition of a system model or control policy into two components:

• Baseline (prior) model or policy: Typically physics-based, analytically derived, or trained offline (e.g., via imitation learning, offline RL, or supervised learning), offering interpretable predictions, stability guarantees, or embedded domain knowledge.
• Online residual corrector: A typically lightweight neural network, linear model, or parametric function, trained or updated in real time to capture only the mismatch (residual) between model output and observed system behavior.

This paradigm can be formalized in control as: $x_{k+1} = f(x_k, u_k) + r(x_k, u_k; \theta)$ where $f$ is the prior model, $r$ is the residual module parameterized by $\theta$ updated online. In policy learning,

$a_{exe} = a_{base} + a_{res}$

with $a_{base}$ from the base policy and $a_{res}$ predicted by the residual policy.

The approach leverages the prior's structural guarantees—such as physical consistency, stability, constraint adherence, or long-horizon prediction—while allocating adaptation to the typically low-dimensional residual, thus improving computational efficiency and sample efficiency (Zhang et al., 2024, Gong et al., 16 Sep 2025).

2. Methodologies and Representative Frameworks

Residual Learning in Model Predictive Control (MPC):

• In adaptive locomotion (Zhou et al., 17 Oct 2025) and vehicle platooning (Zhang et al., 2024), residuals are modeled either via neural networks, random Fourier feature expansions in RKHS, or Q-learning. Systems integrate residual corrections into MPC constraints, updating the residual via online least squares or gradient descent.
• Example: For quadruped locomotion, the dynamics are modeled as $x_{t+1} = f(x_t, u_t) + \Delta(x_t, u_t)$, with $\Delta$ approximated via random Fourier features and updated online for receding-horizon MPC, achieving sublinear dynamic regret against a clairvoyant controller (Zhou et al., 17 Oct 2025).

Koopman-Guided Online Residual Refinement (KORR):

• KORR conditions residual policy on globally predicted next latent states with linear time-invariant Koopman dynamics,

$$z_{t+1}^{base} = A g_\theta(x_t) + B a_{base,t};\quad a_{res,t} = \pi_{res}(z_{t+1}^{base})$$

yielding robust long-horizon predictions and policy extrusion (Gong et al., 16 Sep 2025).

Hybrid Explicit/Implicit Map Representations:

• In large-scale RGB-D mapping, RemixFusion (Lan et al., 23 Jul 2025) and ∇-SDF (Dai et al., 21 Oct 2025) employ explicit coarse grids (TSDF or gradient octree) plus neural residuals for fine geometry, updating both online via buffer-based SGD for mapping completeness and tracking accuracy.

Policy Customization:

• Residual-MPPI (Wang et al., 2024) and Residual Q-learning (Li et al., 2023) employ online residual correction to adapt prior RL/IL policies to new performance requirements at execution, maximizing combined reward functions without retraining the base policy.

Expert Prediction Augmentation:

• Online Residual Learning (ORL) (Vlachos et al., 2024) fuses offline expert predictions with online-learned linear residuals, aggregating corrected predictions via adaptive softmax weights, achieving best-of-both-worlds in trajectory prediction.

3. Training, Update Laws, and Computational Aspects

Across domains, online residual models are optimized via mechanisms tailored for real-time response and data efficiency:

Domain/Framework	Residual Update Mechanism	Typical Update Rate
Vehicle platooning (Zhang et al., 2024)	SGD/Adam on MSE loss, buffer periodicity	~0.4 s (hardware)
Robotics control (Gong et al., 16 Sep 2025)	Alternating PPO and Koopman loss	Episode-based
Sim-to-real compliance (Zhang et al., 2023)	Sequential quadratic programming	Every 0.5–1 s
Predictive tracking (Vlachos et al., 2024)	Recursive least squares	Per timestep
Dense mapping (Lan et al., 23 Jul 2025, Dai et al., 21 Oct 2025)	SGD/Adam on minibatches, rehearsal buffer	1–10 FPS
MPC residual (RKHS) (Zhou et al., 17 Oct 2025)	Projected online gradient descent	200 Hz

Loss functions target either direct prediction error (e.g., MSE), constrained optimization (e.g., complementarity or admittance control constraints), or composite objective balancing smoothness, accuracy, and consistency (as in mapping). Many methods apply online disturbance detectors (Zhang et al., 2024) or context encoders (Nakhaei et al., 2024) to adaptively trigger updates and condition residuals.

Real-time feasibility is explicitly demonstrated in high-frequency domains (10–500 Hz), with residual model sizes and update complexity designed for embedded deployment (e.g., 1000 MACs/update (Chen et al., 21 Jul 2025)).

4. Theoretical Performance Guarantees and Stability

Performance and stability analysis is central to online residual frameworks:

• Global stability via structure: Imposing LTI dynamics in Koopman-guided approaches preserves stability over long horizons, bounded by spectral radius conditions (Gong et al., 16 Sep 2025).
• Sublinear regret: In adaptive MPC with RKHS-residuals, dynamic regret analysis ensures the gap with the optimal clairvoyant controller grows no faster than $O(\sqrt{T})$, vanishing in the limit (Zhou et al., 17 Oct 2025).
• Contraction properties: Residual Q-learning uses the γ-contraction of the Bellman operator for guaranteed convergence of policy customization (Li et al., 2023).
• Passivity guarantees: Admittance control residuals are updated under strict positivity and damping to ensure passive, stable interaction (Zhang et al., 2023).

Empirical ablations confirm less drift, improved extrapolation, and robust recovery from large disturbances compared to purely local or unconstrained nonlinear residuals (Gong et al., 16 Sep 2025), and safety constraints are always enforced by the physics-based backbone in hybrid control.

5. Empirical Benchmarks and Domains of Application

Residual-based online adaptation is validated across a spectrum of control, prediction, and perception tasks:

Application	Performance Gain	Citation
Long-horizon furniture assembly	+3–20 pp success, stable under perturbations	(Gong et al., 16 Sep 2025)
CAV platooning (simulation, hardware)	–58% to –99% error over pure model/NNet	(Zhang et al., 2024)
Sim-to-real manipulation	10/10 success (vs. 3/10 direct transfer)	(Zhang et al., 2023)
Pedestrian trajectory prediction (SDD)	ADE 27.8 px vs. 30.2 (offline), 35.7 (online)	(Vlachos et al., 2024)
Quadruped tracking under large forces	67% improvement over nominal MPC	(Zhou et al., 17 Oct 2025)
Large-scale RGB-D mapping	8.5 FPS, 2.0 cm MAE (vs. 21 cm voxel grid)	(Dai et al., 21 Oct 2025)
Vehicle-trailer navigation	15–30% RMSE reduction in tracking	(Chen et al., 21 Jul 2025)
Autonomous racing agent customization	–8.65% lap time, –23% crash rate	(Wang et al., 2024)

Results consistently demonstrate gains in accuracy, robustness, sample-efficiency and rapid adaptation across both physical and simulated platforms.

6. Limitations, Generality, and Related Controversies

Online residual learning assumes access to interpretable, strong priors and reliable data streams for adaptation. Principal limitations include:

• Residual expressivity: If unmodeled dynamics or disturbances are high-dimensional, a small residual may be insufficient.
• Finite update frequency: Adaptation may lag fast-changing environments, particularly if update rates are low or data is noisy (Zhang et al., 2023).
• Dependence on prior stability: If the baseline is unstable, residual corrections cannot guarantee overall system safety.
• Parameter identifiability: In multi-contact and hybrid systems, decoupling residual corrections from prior error sources may require elaborate parameterizations (Huang et al., 2023).

Across all domains reviewed, true online adaptation is most successful when the prior captures most system structure and the residual is confined to compensating infrequent, low-dimensional discrepancies. The method is generalizable to domains as diverse as power-grid regulation, manipulator control, mapping, vision, and sequential prediction (Zhang et al., 2024, Dai et al., 21 Oct 2025, Vlachos et al., 2024).

7. Future Directions and Research Opportunities

Active areas of research include:

• Global residual-guidance: Leveraging globally structured latent models (Koopman, RKHS) for residual conditioning to further expand robustness (Gong et al., 16 Sep 2025, Zhou et al., 17 Oct 2025).
• Context-encoded adaptation: Using online-inferred context to drive residual corrections for non-stationary or episodically changing dynamics (Nakhaei et al., 2024).
• Divide-and-conquer residualization: Scene factorization and local residual networks enable scaling to very large physical domains (Lan et al., 23 Jul 2025).
• Meta-learning and analytical augmentations: Combining explicit analytical models for contact and force with online meta-learning for rapid geometric generalization (Zhang et al., 2023).

These developments suggest expanding the paradigm to cover high-frequency multi-contact manipulation, legged locomotion, perception-driven control, and continual adaptation in non-stationary environments.

---

Online residual learning frameworks, across their variants (KORR, PERL, Residual-MPPI, ORL, residual mapping), demonstrate the efficacy of fusing domain knowledge with adaptive correction, balancing safety, robustness, computational efficiency, and sample-efficient learning for autonomous, high-performance control and prediction in dynamic, uncertain environments (Gong et al., 16 Sep 2025, Zhang et al., 2024, Zhang et al., 2023, Zhou et al., 17 Oct 2025, Vlachos et al., 2024, Lan et al., 23 Jul 2025, Li et al., 2023, Wang et al., 2024).