An affordable system for the teleoperation of dexterous robotic hands using Leap Motion hand tracking and vibrotactile feedback

Abstract

Using robot manipulators in contexts where it is undesirable or impractical for humans to physically intervene is crucial for several applications, from manufacturing to extreme environments. However, robots require a high degree of intelligence to operate in those environments, especially if they are not fully structured. Teleoperation compensates for this limitation by connecting the human operator to the robot using human-robot interfaces. The remotely operated sessions can also be used as demonstrations to program more powerful autonomous agents. In this article, we report a thorough user study to characterise the effect of simple vibrotactile feedback on the performance and cognitive load of the human user in performing teleoperated grasping and manipulation tasks. The experiments are performed using a portable and affordable bilateral teleoperation system that we designed, composed of a Leap Motion sensor and a custom-designed vibrotactile haptic glove to operate a 4-fingered robot hand equipped with 3-axis force sensors on the fingertips; the software packages we developed are open-source and publicly available. Our results show that vibrotactile feedback improves teleoperation and reduces cognitive load, especially for complex in-hand manipulation tasks.