3D Target Detection of Tractor in Agricultural Scene Based on Lidar

Real time and accurate detection of agricultural machinery in the working scene is an important basis to improve the safety of agricultural machinery driverless and autonomous operation. This paper presents a 3D target detection method of tractor based on GNSS/INS/Lidar/camera data. Firstly, an on-board data acquisition platform based on multi-source sensors is constructed to obtain the tractor image, point cloud and position and attitude data of the acquisition vehicle in the common working scenes of agricultural machinery (machinery depot, tractor road and farmland), the data are time synchronized and space calibrated. Then, distortion correction of point cloud, the point cloud and image are jointly labelled, and the tractor test data set of images, point cloud and GNSS/IMU information is constructed. 643 frames of machinery depot scene data, 321 frames of tractor track scene data, and 320 frames of farmland scene data are obtained. The number of tractors selected is 6250. Finally, according to the characteristics of tractor driving and operation, the region of interest of tractor target detection is defined, the point cloud within its range is extracted, and the ground point cloud is filtered by RANSAC method. Then the data set is input into PVRCNN++ network for training according to the proportion of 5:1 between training set and test set, and finally the tractor 3D target detection model is obtained. The test results show that the detection accuracy of the model for tractors in machinery depot, tractor road and farmland are 90.21%, 87.59%, and 99.85% respectively, the recall rates are 86.72%, 95.13%, and 98.89% respectively, and the F1 scores are 88.43%, 91.20%, and 99.37% respectively. The model can better adapt to the agricultural scene.