A cloud-based face video retrieval system with deep learning

Face video retrieval is an attractive research topic in computer vision. However, it remains challenges to overcome because of the significant variation in pose changes, illumination conditions, occlusions, and facial expressions. In video content analysis, face recognition has been playing a vital role. Besides, deep neural networks are being actively studied, and deep learning models have been widely used for object detection, especially for face recognition. Therefore, this study proposes a cloud-based face video retrieval system with deep learning. First, a dataset is collected and pre-processed. To produce a useful dataset for the CNN models, blurry images are removed, and face alignment is implemented on the remaining images. Then the final dataset is constructed and used to pre-train the CNN models (VGGFace, ArcFace, and FaceNet) for face recognition. We compare the results of these three models and choose the most efficient one to develop the system. To implement a query, users can type in the name of a person. If the system detects a new person, it performs enrolling that person. Finally, the result is a list of images and time associated with those images. In addition, a system prototype is implemented to verify the feasibility of the proposed system. Experimental results demonstrate that this system outperforms in terms of recognition accuracy and computational time.