GraspGraphNet: Graph-Structured Multi-Embodiment Dexterous Grasp Generation

Abstract: Dexterous grasp generation across robot hands is challenging because hands differ in kinematic topology, actuation dimensions, and native command spaces. We introduce GraspGraphNet, a topology-aware grasp generation framework that represents each hand as a URDF-derived kinematic graph and directly generates executable palm poses and joint configurations. GraspGraphNet combines hierarchical object surface encoding, differentiable forward kinematics, and dynamic world-edge message passing to model evolving robot-object interactions. It applies conditional flow matching directly in executable palm-pose and joint-state space, avoiding post-processing optimization, inverse kinematics, and retargeting. Using a shared model trained on Barrett Hand, Allegro Hand, and Shadow Hand, GraspGraphNet achieves an average success rate of 83.48% with 40ms inference time per grasp on a 40-object benchmark. Without retraining, the same model achieves 72.70% success on controlled finger-removal variants, demonstrating robustness to hand-topology variations. These results suggest that graph-structured hand representations can effectively support dexterous grasp generation across robot hands with different kinematic structures. Project: https://lysees.github.io/graspgraphnet-page

• The paper introduces a unified graph-based framework that converts diverse robot hands into URDF-derived kinematic graphs for direct, executable dexterous grasp synthesis.
• It employs dynamic world-edge message passing and conditional flow matching to iteratively refine grasp states with high efficiency and minimal integration steps.
• It achieves high success rates (83.48% in simulation, 91% in real-world) while generalizing across varying kinematic topologies without requiring retraining.

GraspGraphNet: Graph-Structured Multi-Embodiment Dexterous Grasp Generation

Motivation and Context

Dexterous grasp generation in robotic manipulation has long been complicated by the varied kinematic, morphological, and command-space features of multi-fingered robot hands. These hands differ not only in the number of fingers but also in link structure, actuation space, and joint topology, making unified grasp planning challenging. Prior approaches—whether robot-centric, contact-centric, or interaction-centric—have either suffered from embodiment specificity or required post-hoc optimization to produce executable grasps. GraspGraphNet introduces a unified graph-based framework that directly generates robot-native palm poses and joint commands for various hand kinematic topologies by leveraging URDF-derived kinematic graphs, dynamic world-edge message passing, and conditional flow matching. This approach eschews post-processing and retargeting, yielding efficient and robust multi-embodiment grasp generation.

Figure 1: GraspGraphNet represents different dexterous hands as URDF-derived kinematic graphs and generates an executable grasp by evolving an open-hand state $x^{(0)}$ to a final grasp state $x^{(K)}$ with dynamic robot-object world edges.

Methodological Framework

GraspGraphNet is structured around three primary innovations:

Graph-structured hand representation: Each robot hand is converted into a link-joint graph derived from its URDF. Nodes correspond to physical links, and edges encode parent-child joint relationships, encapsulating morphology and actuation in a flexible graph structure. Hand geometries are preserved via per-link embeddings, facilitating information propagation along kinematic connections.

Figure 2: URDF-derived robot hand representations. Physical links form graph nodes, and joints form directed parent-to-child edges.

Dynamic object-robot fusion: Object geometry is captured hierarchically from point clouds, with global and local features extracted at multiple resolutions. During trajectory synthesis, the model builds dynamic world-frame edges between robot links and nearby object surface points, allowing local interaction neighborhoods to evolve as the hand's configuration changes. Message passing over these dynamic edges captures robot-object proximity and contact-relevant geometry throughout inference.

Figure 3: Overview of GraspGraphNet — integration of learned velocity fields over palm pose and joint state, with differentiable forward kinematics, kinematic-tree MPNN, cross-attention, and dynamic world-edge message passing.

Figure 4: State-dependent grasp generation: The grasp state is iteratively refined from an open-hand initialization with state-dependent forward kinematics and dynamic world-edge updates.

Conditional flow matching in executable state space: Instead of predicting intermediate interaction cues requiring subsequent reconstruction, GraspGraphNet generates palm pose and joint configuration directly by learning a conditional velocity field on the grasp-state space. Euler integration of this field, starting from an open-hand initialization, produces the final configuration. The flow-matching objective enables the model to efficiently integrate the trajectory in a small number of steps, improving both inference speed and stability.

Experimental Evaluation

Simulation Results

GraspGraphNet is evaluated against GenDexGrasp, DRO-Grasp, and TRO-Grasp on Barrett, Allegro, and Shadow hands. The evaluation protocol uses Isaac Gym with 40 unseen objects (10 from CMapDataset, 30 from Google Scanned Objects).

Figure 5: Evaluation object set used in simulation. The set contains 40 unseen objects from both CMapDataset and Google Scanned Objects.

GraspGraphNet attains an average success rate of 83.48%, outperforming recent baselines by 5.88% (DRO-Grasp) and 2.48% (TRO-Grasp). Inference time is drastically reduced (average 40 ms per grasp), yielding $14\times$ and $6\times$ speedup over DRO-Grasp and TRO-Grasp respectively, due to direct executable-state generation without post-processing or optimization.

Topology Generalization

To test robustness to kinematic structure changes, finger-ablated variants are constructed for Allegro and Shadow hands, and all models are evaluated without retraining.

Figure 6: Finger-ablated robot hands used for topology generalization. Each variant is represented by its URDF-derived kinematic graph.

GraspGraphNet achieves 72.70% overall success under topology modification, exceeding DRO-Grasp by 15.92% and substantially outperforming contact-centric approaches. This demonstrates the generalization capacity of the graph-based structure, as well as its robustness to changes in link, joint, and graph connectivity. Qualitative results show effective grasps even on topology-altered hands.

Figure 7: Qualitative grasp results on topology-modified robot hands.

Real-World Deployment

GraspGraphNet is tested on a physical setup with a Leap Hand mounted on a UFactory xArm6 and Azure Kinect RGB-D camera. The model, fine-tuned on Leap Hand data, produces executable palm poses and joint configurations from partial point clouds, achieving 91% success across 100 grasp trials.

Figure 8: Real-world experimental setup and test objects. (a) System setup, (b) Test objects.

Ablation Studies

• Flow matching vs. direct regression: Conditional flow matching achieves notably higher success rates than direct regression, highlighting the importance of iterative trajectory refinement in high-dimensional, multimodal grasp generation.
• Object-robot fusion: Removing cross-attention or dynamic world-edge message passing severely degrades performance, indicating that both global context and local geometric interaction modeling are critical.
• Dynamic world edges: Static edge neighborhoods (computed once at initialization) lead to substantial performance drops, underscoring the need for state-dependent interaction updates throughout trajectory synthesis.
• Integration steps ($K$): Increasing $K$ from 1 to 3 substantially improves success rate, but further increments yield diminishing returns and increased inference time. The optimal trade-off is around $K=3$.

Implications and Future Directions

GraspGraphNet’s explicit handling of robot hand topology enables direct, executable grasp generation for diverse embodiments without retraining, representing a significant step toward scalable multi-hand grasp synthesis. Its architecture—centered on URDF-derived kinematic graphs, dynamic message passing, and flow matching—circumvents pitfalls of contact-centric and interaction-centric methods, offering robust transferability and efficiency. Practically, this reduces latency in deployment, enables adaptation to hardware changes (such as finger removal), and supports real-world applications with partial sensor data.

Theoretically, the approach opens avenues for further zero-shot generalization to hands with radically differing morphologies (e.g., underactuated hands, soft robots), as well as extension to dexterous manipulation tasks beyond grasping (e.g., in-hand object reorientation, tool-use). Integration with reinforcement learning or morphologically-aware policy transformers [patel2025get, wu2026dexgrasp] could further enhance cross-embodiment generalization.

Conclusion

GraspGraphNet establishes a topology-aware dexterous grasp generation architecture capable of robust, efficient, and transferable palm pose and joint configuration synthesis for multiple robot hand embodiments. By directly operating on URDF-derived kinematic graphs and modeling dynamic robot-object interactions, it achieves strong numerical results across simulation, topology-variant, and real-world settings. The model’s structure and inference mechanism lay groundwork for scalable and generalizable dexterous manipulation, with practical and theoretical implications for multimodal, cross-embodiment policy learning (2607.11031).