Quadratic Neuron-empowered Heterogeneous Autoencoder for Unsupervised Anomaly Detection

Inspired by the complexity and diversity of biological neurons, a quadraticneuron is proposed to replace the inner product in the current neuron with asimplified quadratic function. Employing such a novel type of neurons offers anew perspective on developing deep learning. When analyzing quadratic neurons,we find that there exists a function such that a heterogeneous network canapproximate it well with a polynomial number of neurons but a purelyconventional or quadratic network needs an exponential number of neurons toachieve the same level of error. Encouraged by this inspiring theoreticalresult on heterogeneous networks, we directly integrate conventional andquadratic neurons in an autoencoder to make a new type of heterogeneousautoencoders. To our best knowledge, it is the first heterogeneous autoencoderthat is made of different types of neurons. Next, we apply the proposedheterogeneous autoencoder to unsupervised anomaly detection for tabular dataand bearing fault signals. The anomaly detection faces difficulties such asdata unknownness, anomaly feature heterogeneity, and feature unnoticeability,which is suitable for the proposed heterogeneous autoencoder. Its high featurerepresentation ability can characterize a variety of anomaly data(heterogeneity), discriminate the anomaly from the normal (unnoticeability),and accurately learn the distribution of normal samples (unknownness).Experiments show that heterogeneous autoencoders perform competitively comparedto other state-of-the-art models.