A simple distributed fuzzy c-means clustering method via the technique of Map-Reduce

The clustering problems are one of the fundamental data analysis tasks in machine learning and data mining. However, many traditional clustering methods cannot directly deal with large-scale data sets as the restriction of memory. In this paper, we develop a simple distributed fuzzy c-means clustering method via the technique of Map-Reduce for large-scale data sets. First, a distributed data dividing method is designed to randomly partition the original data set to several data blocks. Then, we establish a data center determining method through applying fuzzy c-means clustering algorithm in the framework of Map-Reduce. Finally, a distributed clustering method is proposed according to the distance of between each sample and the nearest center. The experimental studies verify the feasibility in clustering accuracy and the parallelism in clustering time.