An end-to-end approach to segmentation in medical images with CNN and posterior-CRF

•A novel CRF method using the learning-based CNN features for medical image segmentation.•The performance of CRF in image segmentation highly depends on its hyperparameters and reference maps.•An end-to-end CRF approach with automatically adapted weights as an RNN process.•Notable improvements compared to the baseline CNN and CRF methods.•Works in difficult cases of three applications, from brain MRI scans to lung CT scans.