AI in Production: Video Analysis and Machine Learning for Expanded Live Events Coverage

As with many industries, TV and video production is likely to be transformed by artificial intelligence (AI) and machine learning (ML), with software and algorithms assisting production tasks that, conventionally, could only be carried out by people. Expanded coverage of a diverse range of live events is particularly constrained by the relative scarcity of skilled people, and it is a strong use case for AI-based automation. This article describes the recent research conducted by the British Broadcasting Corporation (BBC) on the potential production benefits of AI algorithms, using visual analysis and other techniques. Rigging small, static ultra high-definition (UHD) cameras, we have enabled a one-person crew to crop UHD footage in multiple ways and cut between the resulting shots, effectively creating multicamera HD coverage of events that cannot accommodate a camera crew. By working with programme-makers to develop simple deterministic rules and, increasingly, training systems using advanced video analysis, we are developing a system of algorithms to automatically frame, sequence, and select shots, and construct acceptable multicamera coverage of previously untelevised types of events. This paper was published in the proceedings of the International Broadcasting Convention in 2018 [1], and in SMPTE Motion Imaging Journal in 2020 [2].