Monitoring operational status of electromechanical systems using audiovisual information fusion

Complex electromechanical systems have a high probability of failure, which may lead to relatively severe loss in the event of actual malfunction. Moreover, current state monitoring equipment is not universally applicable. Therefore, this paper proposes a method for integrating visual and auditory information to monitor the operating state of an electromechanical device without the aid of built-in sensors. First, the Hilbert–Huang transform is used to decompose the audiovisual waveform in order to obtain the time–frequency information of the signal. Then, the signal characteristics are selected and correlation analysis is adopted to filter the features. Finally, the residual features are input into a generalized regression neural network to obtain the degradation model of the device. In the process of establishing the model, the optimal prediction results are selected by using stochastic resonance to improve the accuracy of the model. Moreover, to reduce the single prediction error, this paper proposes a method for correcting the prediction results by combining historical data. The effectiveness of the proposed method is experimentally verified using a turbojet engine, where the flame fluctuation and sound are specified as the audiovisual information.