UltraCytomic: A cross-scale exploration to improve risk estimation of the malignant thyroid nodule

Initial estimation of the thyroid nodule malignancy mainly relies on the analysis of ultrasound images, but cytology confirmation is always required. Despite clinical decision guides, final criteria evaluation is still prone to subjectivity in both, ultrasound and cytology. Recently, artificial intelligence has been applied to identify malignant thyroid nodules with outstanding results in both image modalities. However, these automatic models have been separately developed either to ultrasound or cytology, ignoring coupled relations which are commonly used by physicians in clinical practice. Malignancy prediction is herein improved by concatenating relevant cytology and ultrasound image features and then used to obtain coupled models. The proposed approach is challenged with 314 patients, coming from two independent databases containing either only ultrasound or coupled ultrasound-cytology images. Basically, the most relevant ultrasound features were selected from the first image database with only ultrasound information (299 patients and 650 thyroid nodules). Afterward, malignancy prediction was improved in a second independent database composed of patient-coupled thyroid/ultrasound images (40 nodules, eight malignant, from 15 patients). The obtained results show that the model trained with coupled information outperformed the one trained with only ultrasound, i.e. accuracy and f1-score were respectively 0.80 and 0.77 when ultrasound and cytology were combined, compared to the respective 0.67 and 0.63 obtained with only ultrasound.