HALO:Closing Sim-to-Real Gap for Heavy-loaded Humanoid Agile Motion Skills via Differentiable Simulation
Abstract: Humanoid robots deployed in real-world scenarios often need to carry unknown payloads, which introduce significant mismatch and degrade the effectiveness of simulation-to-reality reinforcement learning methods. To address this challenge, we propose a two-stage gradient-based system identification framework built on the differentiable simulator MuJoCo XLA. The first stage calibrates the nominal robot model using real-world data to reduce intrinsic sim-to-real discrepancies, while the second stage further identifies the mass distribution of the unknown payload. By explicitly reducing structured model bias prior to policy training, our approach enables zero-shot transfer of reinforcement learning policies to hardware under heavy-load conditions. Extensive simulation and real-world experiments demonstrate more precise parameter identification, improved motion tracking accuracy, and substantially enhanced agility and robustness compared to existing baselines. Project Page: https://mwondering.github.io/halo-humanoid/
Summary
- The paper presents a two-stage differentiable system identification method that improves sim-to-real transfer accuracy for heavy-loaded humanoid motions.
- The paper leverages minimal sensing with joint encoders and constrained trajectory optimization to calibrate both intrinsic and payload-specific dynamics.
- The paper achieves significant reductions in motion tracking errors and enables zero-shot deployment of agile RL policies under severe payload conditions.
HALO: Differentiable System Identification for Sim-to-Real Transfer of Heavy-Loaded Humanoid Motion
Introduction
The HALO framework addresses the significant sim-to-real (sim2real) gap faced in transferring reinforcement learning (RL) locomotion policies to physical humanoid robots under heavy payload conditions. This gap is exacerbated by substantial and structured perturbations in dynamics induced by unknown payloads that significantly alter the robot's mass distribution and inertial properties. Standard approaches leveraging domain randomization (DR) or sampling-based system identification (SysID) are either insufficiently precise or prohibitively costly in terms of required hardware and data. HALO proposes a sensor-minimal, two-stage, gradient-based parameter identification pipeline, built on a differentiable simulator (MuJoCo XLA), which decouples calibration of intrinsic base model inaccuracies from identification of payload-induced dynamic mismatches. This enables robust zero-shot transfer of high-agility RL policies to real hardware.
Figure 1: HALO enables challenging heavy-loaded locomotion, with superior accuracy over DR, including straight-line bidirectional walking, payload attachment, and balancing with constrained single-foot conditions.
Background and Related Work
Existing DR methods seek policy robustness by randomizing simulation parameters within pre-specified ranges. However, these typically yield overly conservative and suboptimal behaviors under massive, structured parameter shifts, such as significant load attachments. Adaptive policy strategies and neural residual model approaches also only partially mitigate sim2real deviations and demand high-quality supervision or lack generalization outside their training domains. Prior SysID frameworks that use torque sensors, external MoCap, or evolutionary sampling-based algorithms (e.g., CMA-ES) are either impractical for complex floating-base systems or inefficient in convergence, particularly under heavy-payload conditions. Recent differentiable simulation paradigms have demonstrated sample-efficient, gradient-based parameter optimization for various robots but remain largely unexplored for the challenging payload-variant floating-base humanoid regime.
HALO Framework
Data Collection with Minimal Sensing
HALO leverages only onboard joint encoders for data collection, circumventing the need for specialized sensors. By mechanically constraining one foot, full-body kinematic reconstruction from joint states is enabled, eliminating drift ambiguities in floating-base estimation. Empirical correction for sensor-induced left-right foot inconsistencies is achieved via constrained trajectory optimization, ensuring physical plausibility.
Figure 2: The HALO pipeline: (a) Data collection under constrained conditions; (b) Trajectory reconstruction from encoders; (c) Two-stage system identification—first base model calibration (no payload), then payload-specific parameter estimation; (d) Zero-shot deployment of RL skills on payload-attached hardware.
Figure 3: Forward kinematics with noisy sensors initially yields unphysical foot-ground contacts; HALO’s physically consistent optimization enforces proper ground adherence.
Two-Stage Gradient-Based System Identification
HALO’s key technical contribution is in decomposing physical parameter identification into a two-stage differentiable optimization:
- Stage 1: Base Model Calibration Using trajectories without load, the full parameter set (masses, centers of mass, inertias, damping, friction) is optimized to reconcile nominal simulation with unsubscribed real-world discrepancies, yielding a calibrated starting model.
- Stage 2: Payload Parameter Identification With the calibrated base model fixed, only payload-relevant (i.e., affected upper limb and torso) mass and CoM coordinates are further optimized, using trajectories under payload conditions. This staged separation mitigates error entanglement, ensuring that payload estimates are not corrupted by preexisting model misalignments.
The identification is performed by minimizing a weighted sum of Cartesian tracking losses (for whole body and upper body) and physical consistency regularizations, using the analytical gradients of system evolution available through the differentiable MuJoCo.
Figure 4: The two-stage HALO method achieves tighter convergence and proximity to payload ground-truth reference values compared to single-stage baselines.
Reinforcement Learning and Policy Transfer
With accurately identified simulation parameters, RL policies are trained for agile motions (e.g., walking, dancing, load carrying) using PPO and motion imitation losses, with only marginal post-SysID domain randomization necessary. This setup enables direct deployment (zero-shot transfer) to the real robot for a spectrum of heavy-loaded locomotion skills.
Experimental Analysis
Superior SysID Under Extreme Payloads
In exhaustive simulation benchmarks against CMA-ES, HALO reliably converges to ground-truth physical parameters even under large dynamic perturbations, whereas CMA-ES exhibits high variance and local minima entrapment under heavy payloads. This demonstrates the advantage of gradient-based differentiable optimization in high-dimensional, structured parameter spaces.
Motion Tracking and Downstream Task Performance
HALO parameters consistently deliver the lowest mean per-joint position error (MPJPE), global MPJPE, and velocity errors on aggressive and steady-state tracking tasks (AMASS, LAFAN1, HuB datasets) compared to wide-range DR and calibrated-mass-only baselines. In real-world experiments on Unitree G1 with 6–10 kg attached payloads, HALO achieves 45–73% reductions in forward position and endpoint errors in walking tasks, and a 73% reduction in angular error on in-place 90° yaw jumps.
Figure 5: Real-world experiments: HALO outperforms baselines on position, endpoint, and angular tracking errors in heavily loaded bidirectional walking and yaw-jumping maneuvers.
HALO achieves 100% success across three high-agility whole-body maneuvers—swallow balancing, side-kick, and roundhouse kicking—while all DR baselines fail and mass-only calibrations are only partially successful.
Practical and Theoretical Implications
HALO’s results have several direct consequences for robust RL-based robot deployment:
- Sensor Minimalism By leveraging only joint encoders and simple, repeatable fixed-foot collection, HALO generalizes to any humanoid platform where external measurements are impractical.
- Structured SysID Scalability The two-stage, gradient-based identification circumvents error entanglement and provides sample-efficient, stable parameter convergence in scenarios where evolutionary methods are fundamentally handicapped.
- Transferability The accurate parameter alignment enables RL policies to operate with minimal additional randomization or adaptation, permitting zero-shot sim2real transfer, including with severe unmodeled payload shifts.
- Robustness/Agility Tradeoff Resolution Unlike DR which biases toward conservatism, HALO policies preserve high agility without sacrificing stability or resilience to nonlinear dynamic shifts.
Theoretical ramifications include highlighting the limits of stochastic DR and derivative-free SysID in structured, high-dimensional sim2real gaps, while showcasing the potential of differentiable simulators for principled, scalable parameter inference.
Future Directions
Potential extensions of HALO include:
- Integration with online adaptive policies leveraging real-time parameter re-identification for non-stationary payloads.
- Application and benchmarking across more varied humanoid platforms and task distributions.
- Generalization to manipulation-intensive whole-body behaviors and multi-contact scenarios with simultaneous complex payload and environment uncertainties.
Conclusion
HALO demonstrates a robust, gradient-based two-stage differentiable system identification and RL framework for sim2real transfer of agile heavy-loaded humanoid motion. Its sensor-minimal, payload-aware methodology yields both strong numerical improvements in SysID and significant empirical gains in real-world robust/versatile skill deployment, setting a new baseline for scalable, practical sim2real solutions in the domain of floating-base, high-DoF legged robotics.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Explain it Like I'm 14
What this paper is about
This paper is about helping humanoid robots move quickly and accurately in the real world even when they’re carrying heavy, awkward, or unknown loads (like backpacks, tools, or weights). The authors show a new way to make what the robot learns in a computer simulator work just as well on the real robot—especially when the robot’s weight and balance change because it’s carrying something.
The main idea, in simple terms
Robots often learn skills in a “video game” version of the world (a simulator). But the real world is messier and different—this difference is called the “sim-to-real gap.” Carrying heavy stuff makes that gap even bigger because it changes the robot’s total mass and balance (its center of mass), so moves that are easy in the simulator can fail in real life.
The authors build a method called HALO that:
- First, figures out the robot’s true physical properties in the real world (like how heavy certain parts are and where their weight is centered).
- Then, trains the robot in a simulator that’s been adjusted to match those real properties.
- Finally, sends the trained skills to the real robot without extra fine-tuning (“zero-shot transfer”).
What questions the paper tries to answer
In everyday language, the paper asks:
- Can we accurately figure out how a robot’s weight and balance change when it carries a heavy load—without fancy sensors?
- Is this “figure-it-out-first” approach better than just training the robot to be tough to all kinds of random changes?
- If we train with a more accurate simulator, will the robot move better in real life, right away?
How the method works (with simple analogies)
Think of it like tuning a video game so it matches real-life physics before you practice tricks:
- Differentiable simulation: The team uses a special physics simulator (MuJoCo XLA) that doesn’t just show you what happens; it also tells you which direction to change numbers (like mass or balance) to make the simulated motion match the real motion. It’s like playing “hot or cold,” but the game points you in the right direction each time, so you adjust faster and more accurately.
- System identification (SysID): This is detective work to uncover hidden facts about the robot—how heavy parts are and where the weight sits. Instead of guessing wildly, the “differentiable” part gives a smart nudge toward the correct answer.
- Two-stage process (why two steps?): 1) Stage 1: Calibrate the “empty” robot (no extra weight). Real robots are never exactly the same as their blueprints due to wear and tiny differences. Calibrating first removes these background errors. 2) Stage 2: Identify the load itself. After the base model is accurate, the method only adjusts the parts affected by the payload (like torso and hands): the load’s mass and where it sits.
- Simple, sensor-minimal data collection: To gather clean data without expensive gear (like motion-capture cameras or torque sensors), the robot stands with one foot clamped to the ground while doing controlled motions. This makes it easier to reconstruct how the body is moving using just the joint angle readings (built-in sensors).
- Training the robot’s skills: After the simulator is updated with the newly identified physical parameters, the robot practices motions (walking, jumping, kicking) in the simulator by imitating reference moves. Because the simulator now matches the real robot, the learned skills are sent to the hardware directly with little or no extra tweaking.
What they found and why it matters
Here are the highlights, explained plainly:
- More accurate “physics detective work”: Compared to a popular guess-and-check method (CMA-ES), their gradient-based approach found the correct masses and balance positions more reliably—especially when the payloads were heavy and made the problem harder.
- Better tracking of motions in simulation: With accurate parameters, the robot’s simulated movements matched the target movements more closely than two common alternatives:
- Training with wide “random changes” (Domain Randomization) made the robot cautious and less precise.
- Only measuring the new weight (without adjusting where that weight sits) helped a bit, but still missed important balance effects.
- Stronger real-world performance with heavy loads:
- Walking straight: The method reduced forward position error by about 45% to 73% compared to baselines.
- In-place 90° turning jump: Reduced orientation error by about 73% compared to the next-best method. The most “robust-by-randomization” approach couldn’t even complete the jump under heavy load.
- Hard moves (like balancing and kicks): Achieved a 100% success rate in tests, while other methods often failed.
- Works without fancy external sensors: The team used only the robot’s built-in joint sensors and a simple one-foot constraint during data collection, making the approach practical and easier to deploy.
Why this matters: When a robot’s load changes (like a person putting on a heavy backpack), its balance and agility can suffer. This method helps the robot stay athletic and precise rather than playing it safe and moving slowly. That’s crucial for real jobs in factories, warehouses, and rescue scenarios.
What this could lead to
- Faster field deployment: Robots can be calibrated quickly for new tools or equipment, then sent to work without lengthy on-site tuning.
- Safer and more capable robots: Accurate physics models let robots attempt fast, complex motions (like jumps and kicks) more reliably, even while carrying weight.
- Broader use beyond humanoids: The same “identify-then-train” idea can be applied to other robots (like quadrupeds or drones) that face changing payloads.
In short, by first making the simulator match reality—and doing it smartly with gradients—the robot can learn powerful skills that transfer smoothly to the real world, even under heavy loads.
Knowledge Gaps
Below is a concise, actionable list of knowledge gaps, limitations, and open questions left unresolved by the paper. These items target what is missing, uncertain, or unexplored, and suggest concrete directions for future research.
- Reliance on a fixed-foot mechanical constraint for data collection limits identifiability of parameters that manifest in free-floating, multi-contact locomotion; investigate identification without mechanical constraints using IMU, proprioception, and contact sensing.
- The method assumes rigid, rigidly-mounted payloads and only estimates mass and CoM; extend to identify full inertia tensors, attachment orientations, non-rigid/flexible payloads, articulated tools, and time-varying payload mass distribution (e.g., sloshing, shifting, swinging).
- Contact model parameters (e.g., foot–ground friction, restitution, compliance) are not identified and are only randomized; study joint identification of contact parameters and their impact on agility and explosive maneuvers.
- Actuation dynamics (motor torque constants, drive gains, latency, saturations, backlash, stiction, thermal effects, battery voltage sag) are not modeled or identified; evaluate joint SysID of actuation parameters and their coupling with inertial parameters under heavy loads.
- Loss uses Cartesian positions only; assess identifiability and performance benefits from incorporating link orientations, velocities, accelerations, torques, and ground reaction forces (when available) into the objective.
- Identification uses open-loop replay of recorded actions with short horizons to avoid divergence; explore closed-loop SysID (e.g., differentiable simulators with feedback controllers), longer-horizon stability, and alignment strategies to reduce trajectory mismatch bias.
- The two-stage procedure restricts stage-2 to mass and CoM only; quantify when full inertia identification is necessary and develop stable formulations for identifying inertia tensors under contacts.
- No formal observability or identifiability analysis; derive conditions and minimal excitation trajectories (and their design) that guarantee unique recovery of targeted parameters on floating-base humanoids.
- Limited discussion of sensitivity to encoder noise and the bias introduced by the foot-height alignment QP; quantify noise robustness and the impact of preprocessing on parameter estimates.
- Data efficiency is claimed but not characterized; measure sample complexity, horizon lengths, and trajectory diversity required for reliable convergence across payload magnitudes and placements.
- Dependence on an initial “wide DR” exploration policy to collect data under load introduces a bootstrapping requirement; develop safe, autonomous excitation policies with formal safety constraints and excitation guarantees.
- The approach calibrates a single base model and then a single payload setting; study rapid re-identification/on-the-fly adaptation when payloads change during deployment or across tasks, including online or continual SysID.
- Uncertainty in identified parameters is not quantified; incorporate Bayesian or ensemble methods to estimate parameter uncertainty and propagate it to robust policy training (e.g., risk-sensitive RL, uncertainty-aware DR).
- Little ablation on the choice of tracking loss weights (e.g., upper-body emphasis) and their effect on biasing parameter recovery; systematically evaluate weighting schemes and regularization strengths.
- Regularization terms constrain parameter drift but their selection is heuristic; investigate principled priors, physical consistency constraints (e.g., positive-definite inertia, triangle inequalities), and constraint-handling in differentiable solvers.
- No comparison with other differentiable SysID frameworks or hybrid methods (e.g., neural residual dynamics + physical parameter identification); benchmark against alternative gradient-based and hybrid approaches.
- Computational performance (time-to-converge, hardware requirements, scalability to higher-dimensional parameters) is not reported; characterize runtime and scalability on practical platforms.
- Generalization across robots and morphologies is untested; validate on different humanoid platforms and quantify portability of the pipeline.
- Real-world experiments cover flat terrain and limited contact conditions; extend evaluation to varied surfaces (slippery, compliant), slopes, stairs, and uneven terrain where contact parameter fidelity is critical.
- Payload configurations are limited (torso + wrists with specific masses); test broader placements (backpack, forearms, end-effectors), asymmetries, and heavier loads approaching actuator limits to map failure modes.
- The impact of minimal DR vs. broader DR after SysID is not fully explored; systematically study the DR “sweet spot” post-identification for robustness without inducing conservativeness.
- Policy performance is reported for select tasks; quantify cross-task generalization (e.g., from locomotion to loco-manipulation) and the effect of parameter misestimation on task families.
- Ground-truth CoM is unavailable in real experiments, and mass alone is compared; devise validation setups with independent ground-truth (e.g., load cells, calibrated backpacks) for CoM/inertia verification.
- The approach assumes known payload attachment links; explore automatic identification of which body segments carry payload and joint estimation of attachment points in the kinematic tree.
- Fragmented optimization strategy (batching trajectories and short windows) may bias estimates; analyze horizon length and fragment selection strategies on bias/variance and convergence stability.
- Safety and hardware wear implications of collecting excitation data under heavy loads are not addressed; develop safety-aware exploration and guardrails for repeated SysID in the field.
Practical Applications
Immediate Applications
Below are deployable applications that leverage HALO’s two-stage, differentiable system identification and minimal-sensor workflow today.
- Payload-aware calibration for humanoid deployment in factories and warehouses (sector: robotics, logistics, manufacturing)
- What: Run a short, fixed-foot routine to identify mass and CoM of attached tools, trays, or boxes; update the robot’s digital twin and deploy zero-shot RL locomotion policies under heavy-load conditions.
- Tools/workflows: “HALO Calibration Routine” (joint-encoder logs + MuJoCo XLA-based optimizer) → parameter update server (URDF/MJCF) → policy selection/deployment with minimal DR → acceptance walk/jump tests.
- Dependencies/assumptions: Access to a safe foot-fixation fixture; payload reasonably rigid; PD-based low-level control; MuJoCo XLA or equivalent differentiable simulator; compute for short gradient-based optimization.
- Rapid tool/changeover procedure for industrial humanoids (sector: robotics, manufacturing)
- What: After swapping or adding a tool/sensor module, perform the HALO two-stage SysID to re-align mass/CoM and enable immediate operation without conservative DR.
- Tools/workflows: Pre-scripted exploration policy with wide DR for data collection; automated parameter update; “green-light” test motions (e.g., bidirectional walking).
- Dependencies/assumptions: Consistent mounting points; ability to constrain one foot during calibration; reliable encoder readings.
- Field-readiness kit for disaster response and public safety (sector: defense, public safety)
- What: Calibrate dynamics for heavy packs, medical kits, or breaching tools before mission; improve gait precision and explosive maneuvers (e.g., jumps/turns) under load.
- Tools/products: Portable “HALO SysID Kit” (mechanical foot clamp, embedded PC with MJX, ROS2 nodes); predefined calibration motions and acceptance metrics.
- Dependencies/assumptions: Short setup time; flat/stable surface for foot fixation; operators trained in a brief data collection procedure.
- Service robots carrying household or hospitality items (sector: consumer, hospitality)
- What: Adapt locomotion to unknown loads (groceries, trays, cleaning gear) with a quick calibration routine, reducing spills and falls.
- Tools/workflows: At-install or pre-shift calibration; update to a library of pre-trained, payload-aware policies.
- Dependencies/assumptions: Payload rigidity and known attach points; safe calibration corner with foot fixture; limited load ranges within tested bounds.
- Sensor-minimal system identification in academic labs and courses (sector: academia, education)
- What: Teach differentiable physics and SysID using only encoders and a fixed-foot constraint—no MoCap/torque sensors required.
- Tools/products: Course labs: datasets, MJX scripts, example policies; reproducible Unitree-G1 recipes; benchmarking tasks (walk, yaw jump).
- Dependencies/assumptions: Licenses for MuJoCo XLA; access to a compatible humanoid; safe fixturing in lab settings.
- Drop-in calibration module for OEMs and integrators (sector: robotics software)
- What: Integrate HALO into vendor toolchains as a “Calibration” wizard, exporting updated dynamics to URDF/MJCF and controller configs.
- Tools/products: ROS2 nodes/plugins; CI hooks in robot configuration pipelines; policy pack conditioned on identified parameters.
- Dependencies/assumptions: API-level access to controllers and model files; QA processes to validate updated parameters with built-in tests.
- Digital twin maintenance for agile motion (sector: software, digital twins)
- What: Keep the digital twin aligned with real hardware after payload changes and wear; reduce sim-to-real drift without broad DR.
- Tools/workflows: Scheduled calibration before high-agility tasks; automatic parameter drift logging and change review.
- Dependencies/assumptions: Stable simulator-hardware mapping; versioning of model parameters; baseline reference trajectories.
- Deployment SOPs and safety checks for heavy-load operations (sector: policy, operations)
- What: Adopt a standard operating procedure requiring payload-specific identification before heavy-loaded maneuvers; define quantitative acceptance thresholds (e.g., forward position and yaw error).
- Tools/workflows: Checklists, automated pass/fail reports, video-backed evidence; repeatable test motions.
- Dependencies/assumptions: Organizational buy-in; alignment with site safety policies; repeatable floor conditions for tests.
Long-Term Applications
These applications require further research, scaling, or integration beyond the current fixed-foot, offline identification paradigm.
- Online, fixture-free payload identification during locomotion (sector: robotics)
- What: Estimate shifting mass/CoM in real time without a fixed-foot constraint (e.g., while carrying sloshing or moving loads).
- Potential tools/products: Contact-state-aware differentiable estimation; incremental gradient updates integrated into the controller.
- Dependencies/assumptions: Robust on-the-fly state estimation; fast differentiable sim or hybrid learned residuals; safety safeguards for online updates.
- Loco-manipulation with unknown, time-varying payloads (sector: robotics, manufacturing)
- What: Seamless handovers, tool use, and transport where payload properties change mid-task; policies conditioned on inferred parameters.
- Tools/workflows: Parameter-conditioned policy libraries; adaptive motion primitives using identified dynamics.
- Dependencies/assumptions: Reliable perception/force cues to detect change; extension beyond mass/CoM to inertia and compliance.
- Cross-platform generalization (quadrupeds, mobile manipulators, exoskeletons) (sector: robotics, healthcare)
- What: Apply two-stage differentiable SysID to other floating-base systems and assistive exoskeletons for load-aware balance and gait.
- Tools/products: Modular pipelines for different robot morphologies; clinical-safe calibration routines for exoskeletons.
- Dependencies/assumptions: Medical approvals for patient-facing systems; adaptation of data collection constraints.
- Regulatory standards for payload-aware autonomy (sector: policy, certification)
- What: Define certification tests requiring calibrated dynamics under specified payload ranges before deployment in workplaces or public spaces.
- Tools/workflows: Standardized test motions and metrics (e.g., E_fpos, E_epos, E_ang); reporting formats and audit trails.
- Dependencies/assumptions: Multi-stakeholder consensus (manufacturers, regulators, insurers); repeatable test environments.
- Predictive maintenance via parameter drift analytics (sector: operations, analytics)
- What: Monitor long-term changes in identified masses/CoMs/damping/friction to flag wear, loose mounts, or sensor faults.
- Tools/products: Trend dashboards; anomaly detection over identification histories; maintenance scheduling.
- Dependencies/assumptions: Regular calibration intervals; stable operating conditions to separate payload vs. wear effects.
- Large-scale fleets with autonomous self-calibration (sector: logistics, field robotics)
- What: Robots perform periodic self-ID and update their digital twins across sites; central server validates and deploys matched policies.
- Tools/workflows: Cloud/edge calibration service; parameter/version management; fleet-wide policy selection and rollout.
- Dependencies/assumptions: Networked infrastructure; secure firmware/config updates; site-safe calibration stations.
- Vision/contact-rich reconstruction to remove fixtures (sector: perception, software)
- What: Replace foot fixation with multi-sensor fusion (vision, IMU, force) for global pose recovery during identification.
- Tools/products: Differentiable perception-simulation pipelines; contact-aware optimization.
- Dependencies/assumptions: Accurate extrinsics and latency management; robust contact modeling in differentiable sim.
- Expanded parameter spaces and environments (sector: robotics R&D)
- What: Identify full inertial tensors, actuator models, and foot-ground friction in varied terrains; extend to non-rigid payloads.
- Tools/workflows: Enhanced regularization and priors; task-specific excitation trajectories to disambiguate parameters.
- Dependencies/assumptions: Richer sensing or carefully designed motions to ensure identifiability; compute scaling.
- Consumer-grade “calibrate-at-home” workflows (sector: consumer robotics)
- What: Simple, guided routines for home humanoids to learn dynamics of baskets, vacuum accessories, or laundry loads.
- Tools/products: App-guided setup; compact foot-locking mat or balance-stand; auto-updated behaviors.
- Dependencies/assumptions: Ease-of-use constraints; robust safety interlocks; variability of home floors and user compliance.
- Integration into co-bot safety envelopes (sector: industrial HRC)
- What: Adjust speed/force limits and safe stopping distances based on identified payload dynamics in human-robot collaboration.
- Tools/workflows: Parameter-informed safety layers; dynamic reconfiguration of ISO-compliant limits.
- Dependencies/assumptions: Harmonization with safety standards; certified runtime monitors.
Notes on General Assumptions and Dependencies
- The present method relies on a short offline calibration with a fixed-foot constraint and assumes payloads are rigidly attached to known links (torso/hands).
- It identifies link mass and CoM positions (not full inertia tensors) and was validated on Unitree G1 with specific load ranges (~6 kg torso, 2 kg per wrist).
- Availability of a differentiable simulator (e.g., MuJoCo XLA) and moderate compute resources is required for gradient-based optimization.
- Low-level PD control and accurate joint encoders are assumed; contact conditions during calibration should be consistent between sim and real.
- Minimal DR is still recommended during policy training, but excessive DR can degrade agility; the identified model allows narrower DR bands for better performance.
Glossary
Below is an alphabetical list of advanced domain-specific terms from the paper, each with a brief definition and a verbatim usage example.
- Adversarial Domain Randomization (DR): A DR variant where environment perturbations are chosen adversarially to stress-test policies during training. "such as curriculum-based or adversarial DR"
- Analytical gradients: Exact derivatives of simulator outputs with respect to inputs/parameters, enabling efficient first-order optimization. "leveraging the analytical gradients over multiple simulation steps"
- Box-constraint function: A penalty that softly enforces variables to stay within given bounds by adding cost outside an interval. "using the box-constraint function:"
- Cartesian space: The 3D Euclidean coordinate space used to measure positions and errors of robot bodies. "measures trajectory discrepancy between the simulated trajectories and the real-world reference trajectories in Cartesian space"
- Center of Mass (CoM): The weighted average position of mass in a body, crucial for dynamics and balance. "altering its total mass, Center of Mass (CoM), and inertia distribution"
- CMA-ES (Covariance Matrix Adaptation Evolution Strategy): A derivative-free evolutionary optimization algorithm often used for black-box problems. "both HALO and CMA-ES methods achieve satisfactory convergence performance"
- Curriculum-based Domain Randomization (DR): A DR strategy that gradually adjusts the difficulty/randomization range over training. "such as curriculum-based or adversarial DR"
- Differentiable simulation: Simulation whose dynamics are differentiable with respect to states, controls, and parameters, allowing gradient-based learning/identification. "Differentiable simulation \cite{related-work-mjx, related-work-brax} has emerged as a paradigm-shifting tool for parameter alignment"
- Differentiable simulator MuJoCo XLA: A gradient-enabled backend of the MuJoCo physics engine supporting automatic differentiation through dynamics. "built on the differentiable simulator MuJoCo XLA."
- Domain Randomization (DR): Training technique that randomizes environment/physical parameters to enhance policy robustness. "A widely adopted strategy to mitigate the sim-to-real gap is Domain Randomization (DR)"
- Evolutionary algorithms: Population-based, derivative-free optimization methods inspired by biological evolution. "sampling-based evolutionary algorithms have been employed to explore high-dimensional parameter spaces"
- Fixed-foot constraint: A mechanical constraint that fixes one foot to remove base pose ambiguity during data collection. "By introducing a fixed-foot constraint, we collect trajectory data"
- Floating-base system: A robot whose base is not rigidly fixed to the world and can move freely (e.g., humanoids). "for floating-based systems like humanoid robots."
- Forward kinematics: Computing the pose of robot links in space from joint angles and the kinematic chain. "via forward kinematics"
- Gradient Descent (GD): An iterative first-order optimization method that updates parameters against the gradient of a loss. "using Gradient Descent (GD):"
- Inertia tensor: A matrix describing how a body's mass is distributed relative to rotational axes, affecting angular dynamics. "avoiding the complexity of optimizing the inertia tensor."
- Joint damping coefficient: The viscous damping parameter at a joint that resists motion proportionally to velocity. "The variables α_d,k and α_f,k represent the joint damping coefficient and static friction of the k-th joint"
- Mean Per-Joint Position Error (MPJPE): An average Euclidean error metric of predicted versus reference joint positions. "Mean Per-Joint Position Error ($E_{\text{mpjpe}$, mm)"
- Motion capture (MoCap): A system that tracks 3D motion of bodies/markers to obtain ground-truth trajectories. "without external measurement systems such as motion capture (MoCap)."
- Motion imitation: Training policies to track or reproduce reference motion trajectories. "We adopt a motion imitation method based on mjlab"
- Multi-Layer Perceptron (MLP): A feedforward neural network composed of fully connected layers. "MLP-based policies are optimized using PPO"
- Payload Parameter Identification: Estimating physical properties (e.g., mass, CoM) of the payload attached to a robot. "Stage 2: Payload Parameter Identification."
- PD controller: A Proportional-Derivative controller that commands torques/positions based on current error and its rate. "tracked via a low-level PD controller."
- Proximal Policy Optimization (PPO): A policy-gradient RL algorithm with a clipped objective for stable updates. "MLP-based policies are optimized using PPO"
- Quadratic program: An optimization problem with a quadratic objective and linear constraints. "we solve a constrained quadratic program"
- Regularization terms: Additive penalties that constrain parameter updates to avoid overfitting or implausible values. "we introduce regularization terms that penalize large changes."
- Residual model: A learned corrective model that augments a nominal simulator to capture unmodeled dynamics. "integrating learned residual model to bridge the gap between idealized simulation and real-world complexity"
- Sim-to-Real gap: The discrepancy between simulated and real-world dynamics/observations that harms transfer. "We interpret heavy-load carrying as a structured sim-to-real gap problem."
- Static friction: The frictional force that must be overcome to initiate motion between contacting surfaces. "The variables α_d,k and α_f,k represent the joint damping coefficient and static friction of the k-th joint"
- System Identification (SysID): Estimating true physical parameters of a system from data to reduce model mismatch. "System Identification (SysID) plays a central role"
- Trajectory-level optimization: Optimizing parameters based on entire rollouts/segments rather than single-step errors. "formulates parameter identification as a trajectory-level optimization problem"
- Wide Range Domain Randomization (WDR): A baseline DR setting that uses broad parameter ranges to encourage robustness. "Wide Range DR (WDR), as illustrated in Table~\eqref{tab:domain_randomization}, (col. 2)."
- Zero-shot transfer: Deploying a learned policy on real hardware without any additional fine-tuning. "enables zero-shot transfer of reinforcement learning policies to hardware"
Collections
Sign up for free to add this paper to one or more collections.