Eye state recognition based on deep integrated neural network and transfer learning

Eye state recognition is widely used in many fields, such as driver drowsiness recognition, facial expression classification, and human–computer interface technology. This study proposes a novel framework based on the deep learning method to classify eye states in still facial images. The proposed method combines a deep neural network and a deep convolutional neural network to construct a deep integrated neural network for characterizing useful information in the eye region by use of the joint optimization method. A transfer learning strategy is applied to extract effective abstract eye features and improve the classification capability of the proposed model on small sample datasets. Experimental results on the Closed Eyes in the Wild (CEW) and Zhejiang University Eyeblink datasets show that the proposed approach outperforms other state-of-the-art methods. In addition, the effects of transfer learning methods with different pretraining datasets on classification accuracy are investigated with the CEW dataset. A driver drowsiness recognition dataset is constructed and used in an experiment to evaluate the effectiveness of the proposed method in driving environments. Experimental results demonstrate that the proposed method performs more stably and robustly than do other methods.