In vivo endomicroscopy enables machine learning-based prediction of responsiveness to neoadjuvant chemoradiotherapy by advanced rectal cancer patients

Probe-based confocal laser endomicroscopy (pCLE) enables in vivo cell-level observation in the colorectal mucosa (CM) during colonoscopy. Assessment of pCLE images is limited by endoscopists’ availability, training, and prevalence of qualitative criteria. Artificial intelligence tools may improve the accuracy of analysis of pCLE movies of the CM contributing for enhanced prognostics. Motiro is an automated unified framework for statistics-based digital pathology of pCLE movies of the CM. Motiro performs a batch mode analysis of pCLE movies for automatic characterization of a tumoral region and its surroundings which enables classifying a patient as responsive to neoadjuvant chemoradiotherapy (neoCRT) or not based on pre-neoCRT pCLE movies. The processing flow is as follows: Motiro builds histograms of fluorescence of all frames; computes the fractal dimension of the contours appearing in frames of videos reporting the tumoral region and its surrounding mucosa; the generated features are feed in Machine Learning (ML) algorithms which aim to predict response to neoCRT. We analyze movies of 47 patients having locally advanced rectal cancer. Accuracy on classification of patients responding or not to neoCRT, based on analysis of images of the tumoral regions or their surrounding areas respectively reach ~0.62 or ~ 0.70. Feature analysis shows that the main contributors for the classification are the fluorescence intensities. We employed a ML framework for predicting whether an advanced rectal cancer patient will respond or not to neoCRT. We demonstrate that the analysis of the mucosa surrounding the tumor region enables better predictive power.