A weights-based accuracy evaluation method for multi class multi-label classifier

Multi label classification is one of the hotspots of Data Mining and Machine learning. An instance may belong to a set of labels simultaneously, and the number of samples which represent by the same instance but with different corresponding labels is imbalanced. The existing evaluation algorithm can not reflect the real performance of one classifier. In order to evaluate the classifier's accuracy more reasonably, a weights-based accuracy evaluation method is proposed. By giving different weight to each sample's classifying result and it can evaluate the real performance of one classifier effectively. The weight is calculated by a formula, according to the lowest weight threshold, degree of imbalance and ratio of the number of samples for each label which one multi-label instance belongs to. Method for calculating these parameters is also proposed in this paper. This evaluation method is used to evaluate the classifying result of multi-label data set, and experiments prove that this method can get a better performance on evaluating the classifier. 1553-9105/ Copyright © 2009 Binary Information Press.