Synergistic learning of lung lobe segmentation and hierarchical multi-instance classification for automated severity assessment of COVID-19 in CT images

•A multi-task multi-instance learning framework is proposed to jointly assess the severity of the COVID-19 patients and segment lung lobe.•A unique hierarchical multi-instance learning strategy is developed to predict the severity of patients in a weakly supervised manner for 3D CT images.•An embedding-level multi-instance learning method is proposed to predict labels beyond the instance-level.•The proposed method achieving promising results in severity assessment in a real-world COVID-19 dataset compared to several state-of-the-art methods.