Enhancing Medical Image Segmentation with Anatomy-aware Label Dependency

Most Neural Networks for organ segmentation are trained to recognize the appearance of the organ, without considering the location of the organ in the body. However, a medical expert would include in their reasoning also the context around the organ. In this work, we propose reproducing this human behavior by enhancing the conventional multi-class segmentation pipeline with additional anatomical information. We apply this concept to a ventral organ segmentation model having a vertebrae label map as additional input, and to a vertebrae segmentation model enhanced by ventral organ information. In both cases, our proposed label dependency approach improved the performance of the baseline models: the dice score (DS) of the ventral organ segmentation improved by more than 3.5 % and the vertebrae identification rate by 1.8%.