Hand sign language recognition using multi-view hand skeleton

•We propose a model for hand sign recognition using SSD, 3DCNN, and LSTM from RGB.•We propose a dataset including 10′000 RGB sign videos.•We build hand skeleton using multi-view projection of 3D hand keypoints.•Our model outperforms state-of-the-art models on NYU and First-Person datasets.•We apply 3DCNN on stacked inputs to get discriminant local spatio-temporal features.