Skip-ST: Anomaly Detection for Medical Images Using Student-Teacher Network with Skip Connections

Anomaly detection (AD) aims to recognize abnormal inputs in testing data when only normal data are available during training. Most AD models perform well on specific datasets but are difficult to generalize to other tasks, especially on medical datasets with high heterogeneity. In this paper, we propose a student-teacher network with skip connections (Skip-ST) which is trained by a novel knowledge distillation paradigm called direct reverse knowledge distillation (DRKD) to realize AD. Skip-ST consists of a pretrained teacher encoder and a randomly initialized student decoder. The output of the teacher encoder's last layer is the input of the student decoder, which aims to recover the multi-scale representation extracted by the teacher encoder. We introduce skip connections between the teacher encoder and student decoder to prevent the student decoder from missing normal information of images at multi-scale. Experimental results show that Skip-ST achieves a 7.95% Area Under the Receiver Operating Characteristic (AUROC) improvement averagely on five challenging medical datasets, outperforming the state-of-the-art AD models. Our code is available at https://github.com/Arktis2022/Skip-TS.