Convoluted mixture of deep experts for robust semantic segmentation

Robust scene understanding of outdoor environments using passive optical sensors is a critical problem characterized by changing conditions throughout the day and across seasons. The perception models on a robot should be able learn features impervious to these factors in order to be operable in the real-world. In this paper, we propose a convoluted mixture of deep experts (CMoDE) model that enables a multistream deep neural network architecture to learn features from complementary modalities and spectra that are resilient to commonly observed environmental disturbances. Our model first adaptively weighs features from each of the individual experts and then further learns fused representations that are robust to these disturbances namely shadows, snow, rain, glare and motion blur. We comprehensively evaluate the CMoDE model against several other existing fusion approaches and show that our proposed model exceeds the state-of-the-art.