Object-Detection from Multi-View remote sensing Images: A case study of fruit and flower detection and counting on a central Florida strawberry farm

Object detection in remote sensing images is one of the most critical computer vision tasks for various earth observation applications. Previous studies applied object detection models to orthomosaic images generated from the SfM (Structure-from-Motion) analysis to perform object detection and counting. However, some small objects that are occluded from the vertical view but observable in raw images from the oblique views cannot be detected in the orthomosaic image, leading to an occlusion issue that cannot be resolved with the traditional orthophoto-based approach. Taking strawberry detection as a case study, the objective of this study is to detect small objects directly from multi-view raw images. Firstly, an object-detection model (Faster R-CNN in this study) was applied to each raw image to identify strawberry fruit and flower objects. Each unique strawberry object on the ground can be detected multiple times in the raw images because images have forward- and side overlap. To find the unique objects from the step one detection results, an improved FaceNet model was proposed to combine the image and position information to calculate the feature distance between those objects, and a clustering algorithm was used to associate the cluster with each unique strawberry using the object distance output from the FaceNet model, from which the final position and number of strawberry fruits and flowers were obtained. Compared with the orthomosaic image alone, this approach using multi-view images effectively solved the occlusion problem and improved overall recognition accuracy of strawberry flowers, unripe fruits, and ripe fruits from 76.28% to 96.98%, 71.64% to 99.09%, and 69.81% to 97.17%, respectively, highlighting the potential of multi-view stereovision (MVS) in small object detection.