Vision-based Picking-Up Control by Robotic Grippers with Belts

This paper presents a vision-based control to pick up an object by a robotic hand. In-hand manipulation, i.e., changing the position and orientation of the grasped object without dropping it, is a challenging task. It is important and necessary for tasks like achieving a stable grasp by translating or rotating the grasped object even though only a part of the object is grasped. However, it is difficult for soft objects like food, because the grasping force needs to be just enough to hold and manipulate the object without crushing it. To tackle this problem, we propose a system which integrates the control of the hand and object detection via a camera. The target robotic hand is configured by two parallel grippers with conveyor belts for manipulation, and is equipped only with a stereo camera as a sensor. From the camera image frames, the three-dimensional position, orientation, and size of the object are calculated. While controlling the grippers and belts, the slippage between belts and the object is estimated based on the difference between their displacements. The validity of the proposed system is verified through the experiments wherein the hand is controlled to pick up objects of varying size and softness. By experimentally evaluating whether the objects have slipped, been crushed, or been manipulated to the target position, it was confirmed that all objects could be picked up with the appropriate force without dropping or crushing them.