DLKNet: Dual Large Kernel Network for Real-Time Semantic Segmentation With Transformer Architecture

The dual-resolution network architecture has proven its effectiveness in real-time semantic segmentation. In recent years, transformer-based networks have achieved great success in visual tasks, prompting researchers to explore how to better transplant the transformer architecture to semantic segmentation. However, due to the relatively high computational cost compared to Convolutional Neural Network based (CNN-based) models, it has been a challenge to apply in real-time semantic segmentation. Fortunately, previous works have shown that using CNN-based models alone can achieve comparable performance to networks using the transformer architecture. In this paper, we build upon the dual-resolution structure of DDRNet and optimize the network further. We propose the LKBlock, which is a transformer-like large kernel block constructed using depthwise separable convolution structures. We use a simple fusion module for merging between different resolution branches and design an auxiliary head for enhancing edge information learning. Through experiments on Cityscapes, our method achieved an mIo U of 76.9% and an inference speed of 103.7FPS, demonstrating the effectiveness of this structure and achieve a better balance between performance and efficiency.