Learning Precise, Contact-Rich Manipulation through Uncalibrated Tactile Skins

Abstract: While visuomotor policy learning has advanced robotic manipulation, precisely executing contact-rich tasks remains challenging due to the limitations of vision in reasoning about physical interactions. To address this, recent work has sought to integrate tactile sensing into policy learning. However, many existing approaches rely on optical tactile sensors that are either restricted to recognition tasks or require complex dimensionality reduction steps for policy learning. In this work, we explore learning policies with magnetic skin sensors, which are inherently low-dimensional, highly sensitive, and inexpensive to integrate with robotic platforms. To leverage these sensors effectively, we present the Visuo-Skin (ViSk) framework, a simple approach that uses a transformer-based policy and treats skin sensor data as additional tokens alongside visual information. Evaluated on four complex real-world tasks involving credit card swiping, plug insertion, USB insertion, and bookshelf retrieval, ViSk significantly outperforms both vision-only and optical tactile sensing based policies. Further analysis reveals that combining tactile and visual modalities enhances policy performance and spatial generalization, achieving an average improvement of 27.5% across tasks. https://visuoskin.github.io/

• The paper proposes the ViSk framework that fuses tactile and visual data to enhance precise, contact-rich robotic manipulation.
• It employs a transformer-based method with low-dimensional magnetic sensors to offer spatially continuous tactile feedback.
• Experimental results demonstrate up to 27.5% improvement over vision-only and optical sensors in diverse real-world tasks.

Learning Precise, Contact-Rich Manipulation through Uncalibrated Tactile Skins

The paper introduces the Visuo-Skin (ViSk) framework, aiming to advance robotic manipulation in precise, contact-rich scenarios by integrating low-dimensional tactile sensing through magnetic skin sensors. The authors address the challenges of traditional visuomotor policy learning, highlighting limitations in reasoning about physical interactions solely with visual data. They explore magnetic skin sensors, which provide low-dimensional, highly sensitive data suitable for seamless integration into robotic platforms.

Key Contributions

The paper proposes ViSk, a framework utilizing AnySkin, a magnetic tactile sensor offering reliable spatially continuous data. This low-dimensional sensor exhibits consistency across different instances and provides a practical, robust alternative to the high-dimensional data from optical tactile sensors that require complex preprocessing for policy learning. ViSk employs a transformer-based architecture where tactile data are treated as additional input tokens alongside visual information.

Experimental Results

The framework’s efficacy is demonstrated across four real-world tasks: plug insertion, USB insertion, credit card swiping, and bookshelf retrieval. ViSk significantly outperforms vision-only models and those relying on optical tactile sensing, showcasing an average performance improvement of 27.5% across tasks. Notably, the framework’s ability to generalize spatially is emphasized, as it shows a marked advantage over purely visual approaches.

ViSk policies exhibit emergent behaviors indicating successful tactile-informed decision-making. For instance, during insertion tasks, the policy leverages tactile feedback around potential contact points, demonstrating better alignment and accuracy.

Comparative Analysis

A salient comparison is drawn between AnySkin and the DIGIT optical tactile sensor. The AnySkin-based policies consistently outperform those using DIGIT sensors, by at least 43% in certain tasks. The study attributes this superiority to AnySkin’s ability to capture low-dimensional, yet effective, tactile feedback crucial for precise manipulation.

Implications and Future Work

The research presents strong implications for the integration of tactile feedback in robotic manipulation, potentially leading to more adaptable and precise robotic systems in everyday applications. The framework circumvents the complications associated with high-dimensional tactile data, offering a practical pathway for real-world deployment.

Future work could explore reinforcement learning techniques to refine Visuo-Skin policies further, potentially enhancing policy performance beyond the current threshold. Additionally, investigation into the role of robot proprioception, which showed mixed results, may provide further insights into improving spatial generalizability in robotic systems.

Overall, this research provides a compelling argument for the integration of simplified tactile sensing into robotic policy learning, highlighting the potential for improved accuracy and reliability in contact-rich tasks.