MacBERT classification model of memory attention mechanism and its application to the power system of EAST neutral beam injection facility.

To provide researcher and operation personnel with recommendations on the condition of equipment so as to ensure the safe, reliable, and smooth operation of the EAST-NBI (Experimental Advanced Superconducting Tokamak Neutral Beam Injection) power system, we propose a memory attention mechanism MacBERT (MLM as correction Bidirectional Encoder Representation from Transformers) text classification model. Firstly, we use MacBERT to generate word vectors containing context, which alleviates the masking differences in pre-training and fine-tuning phases. Secondly, the deep features are further extracted and important parts are highlighted through the memory attention module. Finally, the Linear layer and Softmax layer are used to find the label with the highest predicted probability. Experimental results show that the proposed model performs well in the classification of maintenance records for the EAST-NBI power system, which shows certain research significance.