A joint matrix factorization and clustering scheme for irregular time series data

Key Performance Indicator (KPI) clustering plays an important role in Artificial Intelligence for IT Operations (AIOps) when the number of KPIs is large. This approach can effectively reduce the overhead by dividing KPIs into several classes, then applying the same anomaly detection or prediction model to all KPIs in a class. However, KPI sampling strategies vary depending on the environment in question, which leads to the production of irregular KPIs. Few existing works have considered the clustering of KPIs with irregular sampling. Matrix factorization (MF) is widely applied in low-rank data recovery and can be used to align and fill irregular KPIs. However, the clustering performance after recovering and filling by MF remains unknown. These two problems interact with each other and should therefore be solved together. Accordingly, we formulate the joint MF and clustering problem for irregular KPIs and design an iterative clustering scheme based on MF. This iterative clustering scheme comprises alignment and pre-filling, the loop of clustering, and subclass filling by MF, and can work with two pre-filling methods. Extensive experiments on two real-world datasets show that the iterative clustering scheme can obtain higher normalized mutual information (NMI) than non-iterative clustering, while also consuming less computational time than Dynamic Time Warping (DTW). The two kinds of pre-filling methods each have their advantages on different datasets.