Camera Motion Generation Method Based on Performer's Position for Performance Filming

The role of camera techniques in video quality, a crucial component of visual expression, cannot be overstated. Unlike film or drama productions, live performances offer limited opportunities for reshoots, underscoring the need for meticulous preplanning of camera movements. The proposed method leverages deep neural networks to learn and replicate the camera's positions and postures in response to performers' placements and orientations on stage, thereby mimicking the tacit knowledge of professional camerawork. The method unfolds in two phases: Initially, a network is used to determine camera placements and postures based on the performers' positions and orientations as indicated in the stage script. Subsequently, a second network generates the live camera movements during the performance, factoring in both the performers' placements and orientations and the preliminary camera placements and postures determined by the first network. The architecture of the network incorporates a Transformer that infuses a relative position representation into the input data, proving its ability to more accurately learn camera motion features compared to the standard Transformer.