Learning panoptic segmentation through feature discriminability

•Different requirements of sub-tasks lead to conflicts of feature discriminability.•Separating shared features can enhance feature discriminability of each sub-task.•Region refinement module resolves conflict of feature discriminative between regions.•We report state-of-the-art panoptic segmentation performance on two challenge datasets.