Deep Multidilation Temporal and Spatial Dependence Modeling in Stereoscopic 3-D EEG for Visual Discomfort Assessment

Visual discomfort assessment in stereoscopic three-dimensional (3-D) electroencephalography (EEG) data is challenging. The SOTA research applies deep neural networks to learn temporal information in continuous EEG signals and global spatial information about electrode locations. This work makes the first attempt to jointly deeply model spatio-temporal coupling relationships and select strong dependencies in 3-D EEG data. We explore whether modeling such temporal and spatial dependencies would improve visual discomfort assessment. To address these issues, this work introduces multidilation temporal and spatial dependence-based convolutional neural networks (MTSD) to explore spatio-temporal couplings by 1) learning hierarchical temporal relations within both continuous and interval data and 2) learning and selecting strong spatial dependencies between electrodes (their locations). MTSD captures 1) multidilation temporal relations in continuous and interval receptive fields by a parallel convolution module and 2) spatial dependencies between electrodes in stereoscopic 3-D EEG data by a constrained self-attention module with a copula-based variable dependence strength filter for visual discomfort assessment. Experiments compare five EEG-based deep networks and three MTSD variants. MTSD makes improvement in discriminating visual discomfort by capturing strong spatio-temporal couplings but uses significantly less computational resources.