A Framework to Address the Challenges of Surface Mining through Appropriate Sensing and Perception

The majority of sensing systems in current mining procedures are not optimized to capture the high-fidelity inputs for perception systems. This is due to the lack of accuracy, resolution, update rate, and other shortcomings in the quality and quantity of captured data. High-fidelity input from the multi-modal sensing system is a key component in the development of superior perception technologies. These technologies address critical mining concerns such as performance analysis, progress monitoring, and policy verification for safe operating practices. This paper proposes the non-trivial procedure of proper sensor selection and appropriate deployment of the sensing system to ensure high fidelity multi-modal sensing to serve the above-ground mining applications. Furthermore, the development of perception systems that incorporate this sensing system and machine learning tools is discussed. The mentioned perception systems are mainly used for in-pit visualization, analysis, and decision support applications. Nonetheless, we hope the proposed techniques could be extended to cover complex control applications too. Additionally, we demonstrate the framework for the incorporation of a representative simulated environment, along with the simulated sensors that were chosen through a precise sensor selection analysis, to produce training data for machine learning models. The models are made to be used for processing the real inputs produced by the physical Mobile Sensing Trailer (MST), which is installed in the mining environment. To the best of our knowledge, such simulation environment and its components did not exist or were not accessible before.