Improving Accessibility of Remote Drone Control with a Streamlined Computer Vision Approach

The purpose of this work is to develop a method for classifying hand signals and using the pre-diction output in a drone control algorithm. To achieve this, methods based on Convolutional Neural Networks (CNNs) were applied. The hand signals chosen were the numerical hand signs for one through five for two-dimensional movement with a separate idle signal, and a fist for land. A script was created to automate one minute of training image capture for each class. Transfer learning with PyTorch (Python) was performed using a pre-trained 18-layer residual learning network (ResNet-18). The training process completed in three minutes and 43 seconds with five epochs and a final overall validation accuracy of over 99%. Implemented with the drone control, the classification performed as desired at approximately 60 predictions per second on desktop and 20 predictions per second on a Nvidia Jetson Nano.