A Cross-Dataset Study on the Brazilian Sign Language Translation

Signed communication is an important form of natural language, often less studied, but still relevant. The main question we address in this paper is how to translate Brazilian Sign Language (LIBRAS) implementing Deep Learning networks with limited data availability. Previous studies often use a single dataset, in most cases collected by the authors themselves. We claim a cross-dataset approach would be more adequate to evaluate real-world scenarios. We investigate two methods based on spatial feature extraction. The first one uses pre-trained Convolutional Neural Networks (CNN) and the second one Body Landmark Estimation (skeleton information). A Long Short-Term Memory (LSTM) network is responsible for the sign classification. Our contribution encompasses data curation, alongside providing general guidelines for enhanced generalization.