Integration of stereo vision system calibration and kinematic calibration for an autonomous kiwifruit harvesting system

Stereo vision system and manipulator are two major components of an autonomous fruit harvesting system. In order to raise the fruit-harvesting rate, stereo vision system calibration and kinematic calibration are two significant processes to improve the positional accuracy of the system. This article reviews the mathematics of these two calibration processes and presents an integrated approach for acquiring calibration data and calibrating both components of an autonomous kiwifruit harvesting system. The calibrated harvesting system yields good positional accuracy in the laboratory tests, especially in harvesting individual kiwifruit. However, the performance is not in line with the outcomes in the orchard field tests due to the cluster growing style of kiwifruit. In the orchard test, the calibrations reduce the fruit drop rate but it does not impressively raise the fruit harvesting rate. Most of the fruit in the clusters remain in the canopy due to the invisibility of the stereo vision system. After analyzing the existing stereo vision system, a future visual sensing system research direction for an autonomous fruit harvesting system is justified.