Automatic detection of foreign objects and contaminants in colonoscopic video data using deep learning

Abstract Introduction: In the field of colonoscopy results of automated or supporting procedures using medical image processing are often compromised by contaminants such as blood or stool, but also due to medical instruments. Such objects or contaminations often obscure relevant areas or cause disturbances in various algorithms and also prevent physicians from recognizing relevant information in the video-stream. Hence, automatic detection and segmentation of such contaminations is desirable, by obtaining proper segmentation masks to an algorithm yields the possibility to exclude such areas from analysis. Method: We created a training and validation dataset by collecting (and annotating, where necessary) endoscopic video sequences from various public available datasets and a privat collection. An encoder-decoder deep convolutional neural network was trained with this data to predict segmentation masks for blood, stool, medical instruments and other objects. Results: Depending on the class (blood, stool, ... ), we were able to obtain a mean Dice score in the range between 0.88 to 0.93 on the evaluation dataset. Conclusion: The proposed approach hence allows us providing segmentation masks that are able to reduce problems in the subsequent image analysis and also find regions that were not properly observed.