Gesture-Based Steering Framework for Redundant Soft Robots

Traditional soft robot steering using different devices, such as a mouse or joystick, provides convenient ways to move the robot's tip around in omni-direction, but they may need to be more user-friendly for redundant soft robots as the null space motion can be difficult to teleoperate. In this work, we propose an intuitive approach as a framework to steer a two-segment soft-bodied robot in 3-D space using motile gestures. A lightweight hand-tracking device is employed as the input device to define the position and orientation of the robot tip, and the tip motion can be synchronized from the hand gesture—which is estimated and enhanced by a Kalman filter and low-pass filter with spherical linear interpolation—in real-time. The inverse kinematics (IK) can be rapidly computed based on the convex optimization online. Different hand gestures can be understood to execute the prescribed constrained robot motion. A hardware-in-the-loop simulator and extensive prototype experiments are presented to verify the performance and robustness of the framework. Experiment results show that the master–follower control runs with a steady 11 Hz update rate, and the measured robot's configurations are within 2 mm (for a 100-mm slender manipulator) in either axis in free space steering. This work contributes an intuitive, comprehensible, and low-cost steering framework for redundant soft robots with multimode motion, rapid IK computation, and satisfactory accuracy.