A wearable system for sign language recognition enabled by a convolutional neural network
NANO ENERGY(2023)
摘要
Sign language recognition is of great significance to connect the hearing/speech impaired and non-sign language communities. Compared to isolated word recognition, sentence recognition is more practical in real-world scenarios, but is also more complicated because continuous, high-quality sign data with distinct features must be collected and isolated signs must be identified with high accuracy. Here, we propose a wearable sign language recognition system enabled by a convolutional neural network (CNN) that integrates stretchable strain sensors and inertial measurement units attached to the body to perceive hand postures and movement trajectories. Forty-eight Chinese sign language words commonly used in daily life were collected and used to train the CNN model, and an isolated sign language word recognition accuracy of 95.85% was achieved. For sentence-level sign language recognition, we proposed a method that combines multiple sliding windows and uses correlation analysis to improve the CNN recognition performance, achieving a correct rate of 84% for 50 sign language sentence samples, showing good extendibility.
更多查看译文
关键词
Stretchable strain sensor,Wearable device,Convolutional neural networks,Sign language recognition
AI 理解论文
溯源树
样例
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要