Sustainable Inference of Remote Sensing Data by Recursive Semantic Segmentation – a Flood Extent Mapping Study

In times of climate change and large machine learning models with petabytes of training data, the demand for responsible AI methodologies is more pressing than ever. This is in-particular true for the vast amount of remote sensing data. Its value to explore and inform about earth processes is of paramount importance, especially in times of global warming. Thus, it is nearly ironic that such applications can be the cause of substantial green-house-gas emissions, through energy demands from compute and communication systems. Thus, this study aims at reducing the data transfer between data centers while maintaining near-real time insights from remote sensing data. A recursive inference approach is introduced consisting of three steps: i) Data pyramid preparation in the host data center (sequence of upscaled raster data). ii) The transfer of low-resolution images to the service data center, where a deep-learning model performs a semantic segmentation task, including an uncertainty estimation. Images of higher resolution are then requested and segmented in a recursive fashion, in areas of high uncertainty only. iii) Finally, the merging of the predictions at different resolutions is performed to result in the final pixel-wise segmentation at scale. The method is demonstrated on synthetic and real-world data for a flood mapping task. A U-Net encoder-decoder model is used for the semantic segmentation task, using Monte-Carlo dropout to result in the uncertainty map. The proof-of-concept demonstrated a 35-38% performance gain per transferred pixel compared to high-resolution image segmentation only. Further, we perform a scaling study to estimate the true potential of the recursive inference approach, indicating the potential to reduce a data transfer by up to 98%, considering four hierarchy levels in the data pyramid. With this study, we hope to have contributed a small but important step towards sustainable machine learning.