2000 character limit reached
In-Hand Following of Deformable Linear Objects Using Dexterous Fingers with Tactile Sensing
Published 19 Mar 2024 in cs.RO | (2403.12676v2)
Abstract: Most research on deformable linear object (DLO) manipulation assumes rigid grasping. However, beyond rigid grasping and re-grasping, in-hand following is also an essential skill that humans use to dexterously manipulate DLOs, which requires continuously changing the grasp point by in-hand sliding while holding the DLO to prevent it from falling. Achieving such a skill is very challenging for robots without using specially designed but not versatile end-effectors. Previous works have attempted using generic parallel grippers, but their robustness is unsatisfactory owing to the conflict between following and holding, which is hard to balance with a one-degree-of-freedom gripper. In this work, inspired by how humans use fingers to follow DLOs, we explore the usage of a generic dexterous hand with tactile sensing to imitate human skills and achieve robust in-hand DLO following. To enable the hardware system to function in the real world, we develop a framework that includes Cartesian-space arm-hand control, tactile-based in-hand 3-D DLO pose estimation, and task-specific motion design. Experimental results demonstrate the significant superiority of our method over using parallel grippers, as well as its great robustness, generalizability, and efficiency.
- J. Zhu, A. Cherubini, C. Dune, D. Navarro-Alarcon, F. Alambeigi, D. Berenson, F. Ficuciello, K. Harada, J. Kober, X. Li et al., “Challenges and outlook in robotic manipulation of deformable objects,” IEEE Robot. Autom. Mag., 2021.
- M. Yu, K. Lv, H. Zhong, S. Song, and X. Li, “Global model learning for large deformation control of elastic deformable linear objects: An efficient and adaptive approach,” IEEE Trans. Robot., vol. 39, no. 1, pp. 417–436, 2023.
- C. Wang, Y. Zhang, X. Zhang, Z. Wu, X. Zhu, S. Jin, T. Tang, and M. Tomizuka, “Offline-online learning of deformation model for cable manipulation with graph neural networks,” IEEE Robot. Autom. Lett., no. 2, pp. 5544–5551, 2022.
- M. Yu, H. Zhong, and X. Li, “Shape control of deformable linear objects with offline and online learning of local linear deformation models,” in IEEE Int. Conf. Robot. Autom., 2022, pp. 1337–1343.
- M. Yu, K. Lv, C. Wang, M. Tomizuka, and X. Li, “A coarse-to-fine framework for dual-arm manipulation of deformable linear objects with whole-body obstacle avoidance,” in IEEE Int. Conf. Robot. Autom., 2023, pp. 10 153–10 159.
- X. Huang, D. Chen, Y. Guo, X. Jiang, and Y. Liu, “Untangling multiple deformable linear objects in unknown quantities with complex backgrounds,” IEEE Trans. Autom. Sci. Eng., 2023.
- R. Lee, M. Hamaya, T. Murooka, Y. Ijiri, and P. Corke, “Sample-efficient learning of deformable linear object manipulation in the real world through self-supervision,” IEEE Robot. Autom. Lett., vol. 7, no. 1, pp. 573–580, 2022.
- S. Jin, W. Lian, C. Wang, M. Tomizuka, and S. Schaal, “Robotic cable routing with spatial representation,” IEEE Robot. Autom. Lett., vol. 7, no. 2, pp. 5687–5694, 2022.
- Y. She, S. Wang, S. Dong, N. Sunil, A. Rodriguez, and E. Adelson, “Cable manipulation with a tactile-reactive gripper,” Int. J. Robot. Res., vol. 40, no. 12-14, pp. 1385–1401, 2021.
- X. Jiang, Y. Nagaoka, K. Ishii, S. Abiko, T. Tsujita, and M. Uchiyama, “Robotized recognition of a wire harness utilizing tracing operation,” Robot. Comput. Integr. Manuf., vol. 34, pp. 52–61, 2015.
- J. Chapman, G. Gorjup, A. Dwivedi, S. Matsunaga, T. Mariyama, B. MacDonald, and M. Liarokapis, “A locally-adaptive, parallel-jaw gripper with clamping and rolling capable, soft fingertips for fine manipulation of flexible flat cables,” in IEEE Int. Conf. Robot. Autom., 2021, pp. 6941–6947.
- A. Wilson, H. Jiang, W. Lian, and W. Yuan, “Cable routing and assembly using tactile-driven motion primitives,” in IEEE Int. Conf. Robot. Autom., 2023, pp. 10 408–10 414.
- K. Shaw, A. Agarwal, and D. Pathak, “LEAP Hand: Low-cost, efficient, and anthropomorphic hand for robot learning,” Robotics: Science and Systems (RSS), 2023.
- A. Monguzzi, M. Pelosi, A. M. Zanchettin, and P. Rocco, “Tactile based robotic skills for cable routing operations,” in IEEE Int. Conf. Robot. Autom., 2023, pp. 3793–3799.
- L. Pecyna, S. Dong, and S. Luo, “Visual-tactile multimodality for following deformable linear objects using reinforcement learning,” in IEEE/RSJ Int. Conf. Intell. Robots Syst., 2022, pp. 3987–3994.
- R. B. Hellman, C. Tekin, M. van der Schaar, and V. J. Santos, “Functional contour-following via haptic perception and reinforcement learning,” IEEE Trans. Haptics, vol. 11, no. 1, pp. 61–72, 2017.
- K. Lv, M. Yu, Y. Pu, X. Jiang, G. Huang, and X. Li, “Learning to estimate 3-d states of deformable linear objects from single-frame occluded point clouds,” in IEEE Int. Conf. Robot. Autom., 2023, pp. 7119–7125.
- S. Zhaole, H. Zhou, L. Nanbo, L. Chen, J. Zhu, and R. B. Fisher, “A robust deformable linear object perception pipeline in 3d: From segmentation to reconstruction,” IEEE Robot. Autom. Lett., vol. 9, no. 1, pp. 843–850, 2023.
- W. Yuan, S. Dong, and E. H. Adelson, “Gelsight: High-resolution robot tactile sensors for estimating geometry and force,” Sensors, vol. 17, no. 12, p. 2762, 2017.
- S. Pirozzi and C. Natale, “Tactile-based manipulation of wires for switchgear assembly,” IEEE/ASME Trans. on Mechatron., vol. 23, no. 6, pp. 2650–2661, 2018.
- Z. Yu, W. Xu, S. Yao, J. Ren, T. Tang, Y. Li, G. Gu, and C. Lu, “Precise robotic needle-threading with tactile perception and reinforcement learning,” in Conf. Robot Learn., 2023, pp. 3266–3276.
- F. Ficuciello, A. Migliozzi, E. Coevoet, A. Petit, and C. Duriez, “FEM-based deformation control for dexterous manipulation of 3d soft objects,” in IEEE/RSJ Int. Conf. Intell. Robots Syst., 2018, pp. 4007–4013.
- M. Takizawa, S. Kudoh, and T. Suehiro, “Implementation of twisting skill to robot hands for manipulating linear deformable objects,” in IEEE/RSJ Int. Conf. Intell. Robots Syst., 2016, pp. 945–950.
- S. Zhaole, J. Zhu, and R. B. Fisher, “DexDLO: Learning goal-conditioned dexterous policy for dynamic manipulation of deformable linear objects,” in IEEE Int. Conf. Robot. Autom., 2024.
- GelSight Mini. [Online]. Available: https://www.gelsight.com/gelsightmini/
- P. Beeson and B. Ames, “TRAC-IK: An open-source library for improved solving of generic inverse kinematics,” in IEEE-RAS Int. Conf. Human. Robots, 2015, pp. 928–935.
- S. Qiu and M. R. Kermani, “Precision grasp using an arm-hand system as a hybrid parallel-serial system: A novel inverse kinematics solution,” IEEE Robot. Autom. Lett., vol. 6, no. 4, pp. 8530–8536, 2021.
- D. Kraft, “A software package for sequential quadratic programming,” DLR German Aerospace Center — Institute for Flight Mechanics, Koln, Germany, Tech. Rep. DFVLR-FB 88-28, 1988.
- T. Wimbock, C. Ott, and G. Hirzinger, “Impedance behaviors for two-handed manipulation: Design and experiments,” in IEEE Int. Conf. Robot. Autom., 2007, pp. 4182–4189.
- T. Yoshikawa, “Multifingered robot hands: Control for grasping and manipulation,” Annu. Rev. Control, vol. 34, no. 2, pp. 199–208, 2010.
- S. R. Buss, “Introduction to inverse kinematics with jacobian transpose, pseudoinverse and damped least squares methods,” IEEE J. Robot. Autom., vol. 17, no. 1-19, p. 16, 2004.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.