Automated Full Scene Parsing for Marine ASVs Using Monocular Vision

Perceiving and parsing a maritime scene automatically and in real time is a key task for autonomous surface vehicle navigation. We propose a panoptic segmentation framework that allows end-to-end training and multiple task cascading to meet the navigational challenges of scene parsing in a complex maritime environment. In our framework, the feature extraction backbone is based on Res2Net combined with improved FPN. The fusion network neck adds a mask branch to the latest YOLO detector and embeds a bottleneck attention module. We address possible inference conflict between semantic segmentation and instance segmentation with a panoptic fusion head that resolves conflict using Dezert-Smarandache theory. We also constructed the first maritime scene parsing dataset MarPS-1395, which is completely and fully annotated. MarPS-1395 is the first panoptic segmentation dataset in this field. We validated our model on MarPS-1395 as well as the publicly available dataset to investigate the real-time performance and the accuracy of multitask implementation in panoptic segmentation, which included object detection and classification, instance segmentation, and semantic segmentation. The experimental results also show that our method can robustly accomplish full scene parsing in a complex maritime environment, and achieved a good balance between accuracy of segmentation and speed of computing.