Lymph node metastases detection in Whole Slide Images using prototypical patterns and transformer-guided multiple instance learning

Background: The examination of lymph nodes (LNs) regarding metastases is vital for the staging of cancer patients, which is necessary for diagnosis and adequate treatment selection. Advancements in digital pathology, utilizing Whole-Slide Images (WSIs) and convolutional neural networks (CNNs), pose new opportunities to automate this procedure, thus reducing pathologists’ workload while simultaneously increasing the accuracy in metastases detection. Objective: To address the task of LN-metastases detection, the use of weakly supervised transformers are applied for the analysis of WSIs. Methods & Materials: As WSIs are too large to be processed as a whole, they are divided into non-overlapping patches, which are converted to feature vectors using a CNN network, pre-trained on HE-stained colon cancer resections. A subset of these patches serves as input for a transformer to predict if a LN contains a metastasis. Hence, selecting a representative subset is an important part of the pipeline. Hereby, a prototype based clustering is employed and different sampling strategies are tested. Finally, the chosen feature vectors are fed into a transformer-based multiple instance learning (MIL) architecture, classifying the LNs into healthy/negative (that is, containing no metastases), or metastatic/positive (that is, containing metastases). The proposed model is trained only on the Camelyon16 training data (LNs from breast cancer patients), and evaluated on the Camelyon16 test set. Results: The trained model achieves accuracies of up to 92.3% on the test data (from breast LNs). While the model struggles with smaller metastases, high specificities of up to 96.9% can be accomplished. Additionally, the model is evaluated on LNs from a different primary tumor (colon), where accuracies between 62.3% and 95.9% could be obtained. Conclusion: The investigated transformer-model performs very good on LN data from the public LN breast data, but the domain transfer to LNs from the colon needs more research.