21 Encouraging activity and exercise among children and young people using computer vision and machine learning during gaming

Introduction

Population health and wellbeing is a priority in the UK, with new initiatives that empower children to live healthier lives. Excess weight has also been associated to worse outcomes during the COVID-19 pandemic period, complicated by reduced activity within the confinements of a home environment and coupled by increased screen time with remote classroom practices. As a result, children and young people now interact with computer interfaces in their home environment for education, gaming and healthcare purposes for prolonged periods and in new ways.

Method

There is a growing interest in Natural User Interfaces (NUIs) that use natural hand and body gestures to interact with computers. Advances to these technologies mean that they are now more accurate, easier to use and instead of requiring expensive depth cameras, can be operated using simple webcams. In this study, OpenCV library is used to track user movement by calculating the pixel difference between two frames and create a catalogue of exercises. We use PyTorch exercise recognition model to check the status of the user every 8 frames. These are recognised by using Convolutional Neural Networks (CNNs) with static training from datasets and offer users the option to create personalised exercises.

Result

We present University College London’s (UCL) Motion- Input supporting DirectX: Gestures for at-home exercises. This exercise module can recognise six repetitious static exercises, such as running on the spot, squatting, cycling on an exercise bike, and rowing on a rowing machine using a webcam. This is intended for integrated exercise triggers during gaming in place of a handheld control panel (i.e., jumping to trigger commands), remote coaching for fitness and bespoke treatment plans for physical rehabilitation.

Conclusion

Webcam-based computer vision exercise catalogues using everyday devices like webcams, hold the potential to encourage healthier and more active behaviours during screen-based activities.