Dynamic Vision-Based Machinery Fault Diagnosis with Cross-Modality Feature Alignment

Intelligent machinery fault diagnosis methods have been popularly and successfully developed in the past decades,and the vibration acceleration data collected by contact accelerometers have been widely investigated. In many industrial scenarios, contactless sensors are more preferred. The event camera is an emerging bio-inspired technology for vision sensing,which asynchronously records per-pixel brightness change polarity with high temporal resolution and low latency. It offers a promising tool for contactless machine vibration sensing and fault diagnosis. However, the dynamic vision-based methods suffer from variations of practical factors such as camera position,machine operating condition, etc. Furthermore, as a new sensing technology, the labeled dynamic vision data are limited, which generally cannot cover a wide range of machine fault modes.Aiming at these challenges, a novel dynamic vision-based machinery fault diagnosis method is proposed in this paper. It is motivated to explore the abundant vibration acceleration data for enhancing the dynamic vision-based model performance. A crossmodality feature alignment method is thus proposed with deep adversarial neural networks to achieve fault diagnosis knowledge transfer. An event erasing method is further proposed for improving model robustness against variations. The proposed method can effectively identify unseen fault mode with dynamic vision data. Experiments on two rotating machine monitoring datasets are carried out for validations, and the results suggest the proposed method is promising for generalized contactless machinery fault diagnosis.