Digital Twin Enabled Teleoperated Driving Under Network Delay Using Ego Vehicle Tracking.

Vehicle teleoperation is increasingly seen as a complementary technology to autonomous driving allowing a human teleoperator to be the ultimate guarantor of safety. The major technical hurdles for teleoperated vehicles on public roads are network bandwidth and latency between the vehicle and the teleoperator workstation. A teleoperator requires a sufficiently high update rate, e.g. one update every 0.5 m of travel, and low network latency, e.g. < 300 ms, in order to comfortably control the vehicle. In order to achieve such requirements, we propose to use a real-time digital twin of the vehicular environment based on perception information from the vehicle and a delay-compensating ego vehicle tracking algorithm. Our experiments in a realistic driving simulator show that we can significantly reduce the tracking error under network latency, indicating that higher driving speeds can be achieved with lower update rates and less bandwidth consumption compared to conventional teleoperation solutions based on multiple HD video streams.