Cross-Device Augmented Reality Systems for Fire and Rescue based on Thermal Imaging and Live Tracking.

Augmented Reality (AR) may enhance safety and operational efficiency during firefighting operations. By leveraging modern head-worn AR equipped with embedded sensors and cross-device collaboration, firefighters gain situational awareness. Cross-reality interaction challenges still exist, in low-visibility, high-temperature, life-threatening conditions. This paper introduces two systems targeting distinct firefighting scenarios employing hands-free gaze-based interaction. The first system named Thermal Imaging System addresses fire and rescue operations when thick smoke is a challenge. By integrating a heat-wavelength infrared camera on the Microsoft HoloLens 2, thermal data are streamed directly to the operator’s field of view, enhancing their visibility. Unlike conventional handheld thermal cameras, this approach allows hands-free operation. The second system named Live Tracking System focuses on wildfire, enabling live tracking of each firefighting team member and displaying their locations on a 3D map, accessible remotely on a computer. An AR headset provides the team captain on the field with real-time visual signals signifying team members’ location on the field and in headquarters. A thorough proof-of-concept evaluation with firefighters indicated that the system enabled cross-device collaboration during fire and rescue operations.