Contact-Aware Neural Dynamics Models

• Contact-aware neural dynamics models are techniques that incorporate tactile and contact information to capture discontinuous physical interactions in robotic systems.
• They leverage advanced architectures such as residual correction, structured ODEs, and graph neural networks to embed physical priors and refine dynamic predictions.
• These models enhance state prediction accuracy and sim-to-real transfer while addressing challenges like contact expressivity, computational load, and data efficiency.

Contact-aware neural dynamics models are a class of machine learning approaches designed to capture, predict, and leverage the discontinuous, non-smooth behavior that arises in physical systems involving contacts, impacts, and friction—domains where classical and naive neural dynamics models often fail. These models explicitly incorporate contact information (tactile events, signed distances, or learned contact descriptors) into the dynamic prediction process, thereby enabling accurate simulation, control, and policy optimization for contact-rich robotic tasks. Recent advances incorporate tailored network architectures, structured physical priors, and implicit representations or residual learning strategies, offering scalable solutions for sim-to-real transfer, robust planning, and differentiable simulation in robotics.

1. Mathematical Foundations and Model Classes

Contact-aware neural dynamics models aim to capture both continuous and discontinuous transitions in systems where environmental interaction induces complex events such as impulsive forces, stick-slip transitions, or topological changes in the contact set. Precise mathematical formulations appear across distinct architectures:

• Residual correction models: A base physics simulator ($S_{\rm sim}$) is treated as a prior, with a neural correction term ($\Delta_\theta$) conditioned on contact information:

$$\mathbf{s}_{t+1} = S_{\rm sim}(\mathbf{s}_t, \mathbf{a}_t) + \Delta_\theta(\mathbf{s}_t, \mathbf{a}_t, c_t)$$

where $c_t$ is a binary contact observation, e.g., from tactile sensors (Jing et al., 19 Jan 2026).

• Structured neural ODEs: Physical constraints such as Lagrangian/Hamiltonian structure are embedded, with jump conditions or impulse updates at contact events. Contact complementarity is expressed as:

$$\dot{x} = f_\theta(x, u) + J(x)^\top \lambda,\quad 0 \leq \lambda \perp \Phi(x) \geq 0$$

in which $J(x)$ is a contact Jacobian, $\lambda$ the impulse, and $\Phi(x)$ the signed distance ("gap") function (Hochlehnert et al., 2021, Zhong et al., 2021).

• Implicit and differentiable contact models: Learned signed-distance fields and Jacobians parameterize contact events, with convex optimization-based loss terms ensuring complementarity and maximal dissipation consistent with rigid-body theory (Pfrommer et al., 2020).
• Graph and descriptor field-based models: Geometric and object-centric contact features are encoded via neural descriptor fields or graph neural networks to propagate force/torque effects through contact points (Yang et al., 16 Oct 2025, Yi et al., 15 Sep 2025).
• Latent geometric approaches: Contact flows built on contact geometry and contact Hamiltonians, with learned contactomorphisms, provide global inductive biases for systems with complex energy dissipation and contact topology changes (Testa et al., 22 Jun 2025).

2. Contact Signal Encoding and Network Architectures

Contact information is encoded and utilized differently depending on the architecture and task requirements:

• Binary tactile gating: A one-bit signal ($c_t \in \{0,1\}$) indicates contact presence and gates the residual neural model, enabling the network to condition state transitions on contact events (Jing et al., 19 Jan 2026, Hochlehnert et al., 2021). In simulation, $c_t = 1$ if any fingertip mesh intersects the object mesh; in real data, $c_t = 1$ if normal force on a fingertip exceeds a calibrated threshold.
• Contact prediction and embedding: Neural models predict sequences of future contact events, then embed these as features via MLPs, which are injected via FiLM layers into the main network, particularly in diffusion-based architectures for sequential state prediction (Jing et al., 19 Jan 2026, Okada et al., 2024).
• Implicit geometric representations: Signed distance fields and point cloud encodings (e.g., Neural Descriptor Fields or learned contact maps) are used to provide spatially localized contact information (Yang et al., 16 Oct 2025, Pfrommer et al., 2020). These descriptors are sampled at hand/object keypoints and concatenated into policy or dynamics model input.
• Contact-aware GNNs: Face Interaction Graph Networks (FIGNet-based) introduce action-conditioned world nodes and mesh edges, propagating contact and control information across the tool–environment interface for both motion and force prediction (Yi et al., 15 Sep 2025).
• Latent contact flows: Contact Hamiltonian models use contactomorphisms to map observed state into a latent manifold where contact and dissipation properties are explicit, with ensemble variance controlling geodesic exploration and uncertainty (Testa et al., 22 Jun 2025).

3. Training Objectives, Data Regimes, and Optimization

The construction of contact-aware neural dynamics models requires careful training objectives and diverse data sources:

• Joint loss functions: Models typically optimize for mean-squared prediction error ($\mathcal{L}_{\rm MSE}$) on state transitions, possibly regularized by contact-prediction loss (binary cross-entropy) and auxiliary terms for parameter norm or Lipschitz constraints:

$$\mathcal{L} = \mathcal{L}_{\mathrm{cnt}} + \lambda \mathcal{L}_{\mathrm{MSE}}$$

(Jing et al., 19 Jan 2026).

• Simulation and real-robot co-training: Large simulated datasets are used for pre-training; a smaller set of real-world trajectories is collected for fine-tuning, balancing diversity and accuracy. Mini-batch mixing or explicit loss weighting can prioritize real data during co-training (Jing et al., 19 Jan 2026, Okada et al., 2024).
• Physics-inspired regularization: In structured ODE and contact models, explicit losses enforce complementarity ($0 \leq \lambda \perp \Phi \geq 0$), non-penetration, and maximal dissipation via convex quadratic programs. Differentiation through such optimization, using KKT conditions or CvxpyLayers, ensures tractability (Pfrommer et al., 2020, Zhong et al., 2021).
• Diffusion and score-based learning: When modeling highly non-linear, multi-modal contact-induced trajectories, denoising diffusion models are used to capture the conditional distribution over force trajectories, with training based on score-matching losses (Okada et al., 2024, Jing et al., 19 Jan 2026).
• Contact supervision and touch feedback: Touch or contact labels are critical for disambiguation of discontinuous state jumps. Access to binary contact during training substantially improves learning identifiability and reduces state forecast error, whereas models lacking such supervision may conflate impulsive and smooth dynamics (Hochlehnert et al., 2021).

4. Empirical Benchmarks and Performance Evaluations

Contact-aware neural dynamics models have been benchmarked on a diverse array of contact-rich robotic tasks and physical datasets:

• State prediction and pose accuracy: Metrics typically include MSE on object pose, ADD-S AUC (average distance of model points), and root mean squared error (RMSE) over long rollouts (Jing et al., 19 Jan 2026, Hochlehnert et al., 2021). Performance gains of 20–30% in MSE and ADD-S have been observed when including contact signals; state-of-the-art models achieve MSEs as low as 0.0058 and ADD-S of 88% in sim-to-real settings (Jing et al., 19 Jan 2026).
• Task success rates: In manipulation tasks such as in-hand reorientation, object stacking, dynamic recovery, or insertion, contact-aware models significantly increase final task success—often by over 20% compared to contact-agnostic baselines (Jing et al., 19 Jan 2026, Yang et al., 16 Oct 2025, Yi et al., 15 Sep 2025).
• Force and torque prediction: Unified models incorporating contact information improve force/torque forecast accuracy by up to 3$\times$ over classical simulators in both sim and hardware evaluations (Yi et al., 15 Sep 2025).
• Sample and data efficiency: Contact-aware models achieve state-of-the-art accuracy with much less real data—down to a few dozen trajectories—by leveraging inductive biases, multimodal encoding, and prior simulators (Hochlehnert et al., 2021, Pfrommer et al., 2020).
• Generalization and robustness: Contact descriptors (e.g., Neural Descriptor Fields) enable zero-shot policy transfer to unseen object shapes; sim-to-real aligned models remain robust under domain shift and partial drift in external pose trackers (Yang et al., 16 Oct 2025, Jing et al., 19 Jan 2026).

5. Applications in Robotic Policy Refinement and Sim-to-Real Alignment

A central motivation for contact-aware neural dynamics models is deployment in policy learning and sim-to-real transfer loops:

• Forward-model-based policy evaluation: Learned models serve as forward simulators to estimate the real-world success of candidate policies, filtering poor-performing policies before physical deployment and reducing risk (Jing et al., 19 Jan 2026).
• Iterative sim-to-real adaptation: Contact-aware models enable iterative cycles: train control policies in simulation; evaluate and select top candidates via the refined model; deploy, collect more real data; update the model; and retrain policies, leading to progressive sim-to-real convergence (Jing et al., 19 Jan 2026).
• Control and planning via differentiable simulation: Contact-aware neural ODEs, diffusion models, and GNN-based predictors provide differentiable rollouts amenable to gradient-based planning, trajectory optimization, and variable-impedance search (Zhong et al., 2021, Okada et al., 2024, Yi et al., 15 Sep 2025).
• Adaptive recovery and generalization: Policies leveraging contact descriptors achieve higher task success in dynamic recovery, manipulation, and downstream assembly, especially under object geometry variation and unmodeled disturbances (Yang et al., 16 Oct 2025).

6. Limitations, Open Questions, and Comparative Insights

Despite significant progress, several limitations persist:

• Contact signal expressivity: Binary contact encoding omits force magnitude and direction; models relying only on 1-bit signals may miss subtleties such as slip or stick/frictional transitions (Jing et al., 19 Jan 2026).
• Pose tracking and drift: Reliance on external pose trackers introduces noise and drift; model accuracy degrades for long-horizon rollouts with frequent contact transitions (Jing et al., 19 Jan 2026).
• Computational demands: Some architectures, particularly those using diffusion or large GNNs, require GPU acceleration for real-time performance, and inference time may lag behind optimized classical simulators (Okada et al., 2024, Yi et al., 15 Sep 2025).
• Physical interpretability: Black-box models may lack interpretability with respect to the underlying complementarity structure or energy evolution. Efforts embedding structured physics (e.g., contact Hamiltonians, SOCP-based contact resolution) maintain physical interpretability but may add training or design complexity (Pfrommer et al., 2020, Zhong et al., 2021, Testa et al., 22 Jun 2025).
• Data requirements and uncertainty: High-dimensional or contact-rich environments may demand larger datasets or more expressive capacity, risking overfitting. Ensemble and uncertainty-aware latent contact models address some of this by steering rollouts toward manifold regions observed in data (Testa et al., 22 Jun 2025).
• Extension to softer contacts and complex friction: Most models target rigid-body, impulsive contacts. Extensions to deformable bodies or soft contacts—beyond frictional point contacts—remain an active area (Zhong et al., 2021, Testa et al., 22 Jun 2025).

7. Comparative Table: Core Model Classes and Characteristics

Model/Approach	Contact Representation	Physical Priors / Solver	Application Domain
Residual Correction (Jing et al., 19 Jan 2026)	Binary touch, tactile	Simulator + neural residual	Manipulation, sim-to-real
Physically Structured ODE (Hochlehnert et al., 2021)	Binary touch, gap $\Phi$	Lagrangian VI, symplectic integrator	Rigid body, impacts
Differentiable Contact Model (Zhong et al., 2021)	Gap $\Phi_C$, SOCP impulse	Lagrangian/Hamiltonian NN + SOCP	Bouncing, friction, planning
Descriptor Field / GNN (Yang et al., 16 Oct 2025, Yi et al., 15 Sep 2025)	NDF or mesh-based face contacts	PointNet/GNN aggregation	Dynamic grasp, insertion
Diffusion-based (Okada et al., 2024, Jing et al., 19 Jan 2026)	Conditioning on contact	Diffusion, U-Net denoiser	Variable-impedance control
Contact Flow (Testa et al., 22 Jun 2025)	Ensemble contactomorphism	Contact Hamiltonian geometry	General dissipative systems

These models demonstrate that leveraging explicit or learned contact representations within neural architectures—complemented by physically meaningful inductive biases and tailored loss functions—substantially improves data-efficiency, physical realism, and sim-to-real applicability in contact-rich dynamical systems. Cited models and findings delineate the frontier in scalable, robust learning for robotic manipulation and control in non-smooth domains.