Animation-Conditioned Policies with Physical Constraints

• The paper presents a framework that integrates animation cues with robust physical constraint modeling to generate safe, expressive robot motions.
• It utilizes augmented state representations and dual latent conditioning to balance stylistic fidelity with strict physical limits.
• Empirical results demonstrate improved hardware transfer, reduced tracking errors, and effective thermal and impact control for reliable operation.

Animation-conditioned policies with physical constraints are a class of control methodologies that integrate stylized or data-driven animation references into robotic policy learning while rigorously enforcing physical limitations inherent to real-world hardware or simulation environments. This paradigm enables the generation and execution of expressive, plausible, and safe motions in robots and virtual characters, allowing for both stylistic fidelity and operational soundness.

1. Formal Policy Structure and State-Action Representation

Animation-conditioned policies accept augmented observation spaces that systematically encode both proprioceptive and exogenous cues. Representative formulations, such as in Olaf’s hardware walking system, define the state vector at time $t$ as:

$$s_t = \bigl(p_t^\mathcal{P},\,\theta_t^\mathcal{P},\,v_t^\mathcal{R},\,\omega_t^\mathcal{R},\,q_t,\dot q_t,\, a_{t-1},\,a_{t-2},\,T_t,\phi_t\bigr)$$

where the components correspond to root pose/velocity, joint states, prior actions (for temporal smoothing), actuator temperatures, and gait phase indicators (Müller et al., 18 Dec 2025). The action is typically a vector of per-joint position setpoints or target torques for local PD controllers. Policy conditioning is implemented via high-level animation cues (e.g., puppeteering commands) that are transformed, through motion generators, into kinematic references for imitation tracking.

In latent-conditioned approaches, as with "Latent Conditioned Loco-Manipulation Using Motion Priors," low-level policies are conditioned on continuous latent variables $z$ representing skill manifold locations, modulating style and execution within physical bounds (Stępień et al., 19 Sep 2025).

2. Animation Conditioning and Motion Prior Integration

Animation conditioning is realized by supplying reference trajectories, sampled clips, or latent encodings derived from stylized animation data (mocap, artist-authored keyframes, or procedural generators):

• Explicit kinematic targets ($x_t = f(\cdots)$) are computed and fed to the policy for real-time tracking of character pose and style (Müller et al., 18 Dec 2025).
• Latent embedding models train skill-encoders—e.g., von Mises–Fisher posteriors on transition pairs $(s_t, s_{t+1})$—to map high-dimensional motion priors into compact control codes $z$ (Stępień et al., 19 Sep 2025).
• RLAnimate conditions its behaviour model on a dual-latent structure: one latent for task objectives, another for animation behaviour, with a stochastic posterior incorporating motion-clip-derived "ideal" descriptions (Gamage et al., 2021).
• Diffusion-based policies (e.g. PDP) maintain action-conditioned Markov chains of noise and denoising networks, ensuring direct alignment with demonstration trajectories while enabling multi-modal skill generation (Truong et al., 2024).

3. Physical Constraint Modeling and Enforcement Techniques

Physical constraints are imposed at multiple layers of the policy learning and rollout architecture:

• Thermal Constraints: In Olaf’s embodied policy, actuator temperatures $T$ are modeled by a first-order differential system,

$$\dot{T} = -\alpha(T-T_{\text{ambient}}) + \beta \tau^2$$

and controlled via barrier-function rewards penalizing violations of $T \le T_{\max}$ (Müller et al., 18 Dec 2025).

• Impact/Noise Control: Acoustic artifacts from foot contacts are minimized by penalizing high vertical velocity changes at stance transitions, reducing ground impact noise (Müller et al., 18 Dec 2025).
• Contact and Torque Limits: All systems enforce joint torque and kinematic boundaries through simulation invariants or explicit constraint termination (e.g., the CaT framework, which stochastically ends rollouts on ground reaction force excess (Stępień et al., 19 Sep 2025)).
• Contact/Balance: Simulators like MuJoCo enforce no interpenetration, friction cone constraints, and complementarity, typically handled at each simulation timestep (Truong et al., 2024).
• Bounded Actions: Beta-distributed action heads confine predicted rotations to joint-legal intervals, eliminating constraint-violation risk in RLAnimate (Gamage et al., 2021).

Table: Physical Constraint Mechanisms Across Key Systems

System	Constraint Types	Enforcement Method
Olaf (Müller et al., 18 Dec 2025)	Temperature, Impact, Joint	Barrier rewards, penalty terms
LaCoLoco (Stępień et al., 19 Sep 2025)	Torque, Kinematic, GRF	Simulator, CaT stochastic termination
PDP (Truong et al., 2024)	Torque, Contact, Friction	Simulator (MuJoCo)
RLAnimate (Gamage et al., 2021)	Joint-angle, Smoothness	Beta-action heads, imitation loss

4. Reward Composition and Optimization Objectives

Policies are trained with composite reward functions balancing animation fidelity, smoothness, regularization, joint-limit observance, and physical constraint adherence. For Olaf,

$$r_t = r_t^{\rm imitation} + r_t^{\rm regularization} + r_t^{\rm limits} + r_t^{\rm impact}$$

with terms representing tracking of animator-driven motion, penalization of excessive torques/accelerations, penalty for joint/temperature/foot collision violations, and impact sound control (Müller et al., 18 Dec 2025). Weighting constants are empirically tuned per constraint and operational regime. In constrained latent-policy learning, episodic returns are modulated by termination penalties when physical constraints are violated (probabilities computed from constraint violation magnitudes, e.g. ground reaction force excess) (Stępień et al., 19 Sep 2025).

Diffusion-discriminator-based imitation rewards (DRAIL) supplant GAN-style divergence metrics with noise-conditioned denoising loss, yielding improved match with reference transitions (Stępień et al., 19 Sep 2025); similar denoising objectives are utilized in PDP (Truong et al., 2024).

5. Policy Architectures and Training Protocols

Neural policy architectures span multi-layer perceptrons for actor/critic components (512 units, 3 layers in Olaf), transformer-based score models for diffusion policies (6-layer encoder-decoders in PDP), and recurrent dual-latent models for behaviour-task disentanglement (GRU cells + MLPs in RLAnimate) (Müller et al., 18 Dec 2025, Truong et al., 2024, Gamage et al., 2021).

Training leverages PPO for RL agents (clip ratio 0.2, lr $3 \times 10^{-4}$, $\approx$32k batch, multi-thousand environments (Müller et al., 18 Dec 2025, Stępień et al., 19 Sep 2025)), extensive domain randomization for sim-to-real transfer (input/output noise, friction/mass randomization) (Müller et al., 18 Dec 2025, Stępień et al., 19 Sep 2025), and supervised imitation via denoising or batch rollout aggregation for diffusion/BC policies (Truong et al., 2024).

Empirical learning durations for physically embodied characters (Olaf) are on the order of 2 days on RTX 4090 for 100k PPO iterations; sample efficiency is further characterized in RLAnimate ($\approx$0.5M episodes versus $\gg$10M for DeepMimic RL) (Müller et al., 18 Dec 2025, Gamage et al., 2021).

6. Transfer to Hardware and Empirical Results

Simulation-to-hardware transfer necessitates robust constraint generalization and sensor noise resilience. Olaf’s system achieves mean joint tracking errors of 3.87–4.02°, maintains actuator thermal limits (head-pitch now <80°C versus 100°C under naive baseline), and reduces impact noise by 13.5 dB (hardware) (Müller et al., 18 Dec 2025). Latent-conditioned loco-manipulation controllers on quadruped hardware attain 5.6 cm mean error with 0% falls and marked GRF violation reduction (Stępień et al., 19 Sep 2025).

Diffusion-policy validators demonstrate high performance on perturbation recovery (no fall under strong pushes), universal motion tracking (local/global MPJPE, velocity, acceleration errors), and physics-based text-to-motion synthesis success rates, matching or exceeding prior VAE/MLP methods (Truong et al., 2024). RLAnimate achieves imitation rates exceeding 99% and smoothness above 98.5%, with rigorous ablation proving the necessity of split latent dynamics and imitation regularization (Gamage et al., 2021).

7. Theoretical and Practical Significance

Animation-conditioned policies with explicit physical constraint modeling are foundational for producing robust, expressive real-world robot behaviors and high-fidelity virtual character animations. These systems address the dual challenge of stylistic generalization—capturing artist intent, demonstration data, or procedural style—and verifiable safety/feasibility under the physics of actuation, temperature, contact, and hardware wear.

Current results confirm substantial advances in hardware transferability, sample efficiency, stylistic interpolation, and operational longevity, positioning these methods as central frameworks for physically grounded animation and multi-skill robotic control (Müller et al., 18 Dec 2025, Stępień et al., 19 Sep 2025, Truong et al., 2024, Gamage et al., 2021). A plausible implication is continued refinement in hierarchical latent-conditioned policy models, further quantitative evaluation on hardware, and expansion toward more complex character morphologies and social-interaction behaviors.