Image Segmentation Models as a New Tool to Monitor Disease Risks in Changing Environments

Abstract Background: In the near future, mosquito-borne diseases may expand in new sites due to changing temperatures and rainfall patterns caused by climate change. Therefore, there is a need to use recent technological advances to improve vector surveillance methodologies. UAVs (Unmanned Aerial Vehicles) have been used to collect high-resolution imagery (2-10cm/ pixel) to map detailed information on mosquito habitats and direct control measures to specific areas. Open-source foundation models such as the Meta AI Segment Anything Model (SAM) can facilitate the automation of the manual digitalization process. This model can assist in extracting the features of interest in images of the diverse domains. We evaluated the performance of the Samgeo package, based on SAM, since it has not been applied to analyse remote sensing data for epidemiological studies. Results: We tested the identification of two landcovers of interest: water bodies and human settlements. Different drone platforms acquired imagery across three malaria-endemic areas: Africa, South America, and Southeast Asia. The input was provided through manually located point prompts and text prompts associated with the classes of interest to guide the segmentation and compare the performance in the different geographic contexts. The results indicate that point prompts can significantly decrease the human effort required for annotations. Nevertheless, the performance of text prompts was closely dependent on each object's features and landscape characteristics, resulting in varying performance. Conclusions:Recent models such as SAM can potentially assist manual digitalization in vector control programs, quickly identifying key features when surveilling an area of interest. However, it still relies on the user manual prompts and corrections to obtain the gold standard segmentation and specific tests and evaluations if intended to be used in rural areas.