A Smart Camera with Integrated Deep Learning Processing for Disease Detection in Open Field Crops of Grape, Apple, and Carrot

ABSTRACTDowny mildew (Plasmopara), apple scab (Venturia inaequalis), and Alternaria leaf blight are endemic diseases that affect crops worldwide. The diseases can cause severe losses in grapes, apples and carrots when not detected and treated in an early stage. The European Union Horizon 2020 OPTIMA project aimed to improve disease detection in the open field with an automated detection system as part of an integrated pest management (IPM) system. In this research, we investigated the automated detection of downy mildew in grape, apple scab in apple and Alternaria leaf blight in carrot, using a deep convolutional neural network (CNN) on RGB color images. Detections from the CNN served as input to a Decision Support System (DSS), to precisely locate and quantify the disease, so that appropriate and timely application of plant protection products could be recommended. The focus of our study was on a smart camera implementation with integrated deep‐learning processing in real‐field conditions. The question was whether the deep learning model, when trained on images of disease symptoms recorded in conditioned circumstances, can also perform on images of disease symptoms recorded in field conditions. This type of evaluation is called open‐set evaluation, and so far it has received little attention in plant disease detection research. Therefore, the goal of our research was to evaluate the performance of a deep learning model in an open‐set evaluation scenario in commercial vineyards, orchards, and open fields. The model's performance in the open‐set scenario was compared to its performance in the closed‐set scenario, which involved evaluating the trained model on images similar to those used for model training. Our results showed that the model's performance in the closed‐set scenario with F1 scores of 66.3% (downy mildew), 45.1% (apple scab), and 42.1% (Alternaria) was notably better than in the open‐set scenario, with F1 scores of 34.8% (downy mildew), 5.5% (apple scab) and 4.2% (Alternaria). Uniform Manifold Approximation and Projection (UMAP) analysis proved the significant difference between the open‐set and closed‐set data sets. Our result should encourage other researchers to carry out similar open‐set evaluations to get realistic impressions of their model's performance under field conditions. A subset of our image data set has been made publicly available at https://doi.org/10.5281/zenodo.6778647.