Triplet Learning for Chest X-Ray Image Search in Automated COVID-19 Analysis

Chest radiology images such as CT scans and X-ray images have been extensively employed in computer-assisted analysis of COVID-19, utilizing various learning-based techniques. As a trending topic, image retrieval is a practical solution by providing users with a selection of remarkably similar images from a retrospective database, thereby assisting in timely diagnosis and intervention. Many existing studies utilize deep learning algorithms for chest radiology image retrieval by extracting features from images and searching the most similar images based on the extracted features. However, these methods seldom consider the complex relationship among images (e.g., images belonging to the same category tend to share similar representations, and vice versa), which may result in sub-optimal retrieval accuracy. In this paper, we develop a triplet-constrained image retrieval (TIR) framework for chest radiology image search to aid in COVID-19 diagnosis. The TIR contains two components: (a) feature extraction and (b) image retrieval, where a triplet constraint and an image reconstruction constraint are embedded to enhance the discriminative ability of learned features. In particular, the triplet constraint is designed to minimize the distances between images belonging to the same category and maximize the distances between images from different categories. Based on the extracted features, we further perform chest X-ray (CXR) image search. Experimental results on a total of 29, 986 CXR images from a public COVIDx dataset with 16, 648 subjects demonstrate the effectiveness of the proposed method compared with several state-of-the-art approaches.