Improving Automatic Endoscopic Stone Recognition Using a Multi-view Fusion Approach Enhanced with Two-Step Transfer Learning

This contribution presents a deep-learning method for extracting and fusing image information acquired from different viewpoints, with the aim to produce more discriminant object features for the identification of the type of kidney stones seen in endoscopic images. The approach was specifically designed to mimic the morpho-constitutional analysis to visually classify kidney stones by jointly using surface and section images of kidney stone fragments. The model was further improved with a two-step transfer learning approach and by attention blocks to refine the learned feature maps. Deep feature fusion strategies improved the results of single view extraction backbone models by more than 6% in terms of accuracy of the kidney stones classification.