ClustKNN: A Highly Scalable Hybrid Model- & Memory-Based CF Algorithm

Collaborative Filtering (CF)-based recommender systems are indispensable tools to find items of interest from the unmanageable number of available items. Moreover, com- panies who deploy a CF-based recommender system may be able to increase revenue by drawing customers' attention to items that they are likely to buy. However, the sheer num- ber of customers and items typical in e-commerce systems demand specially designed CF algorithms that can grace- fully cope with the vast size of the data. Many algorithms proposed thus far, where the principal concern is recom- mendation quality, may be too expensive to operate in a large-scale system. We propose ClustKnn, a simple and intuitive algorithm that is well suited for large data sets. The method first compresses data tremendously by build- ing a straightforward but ecient clustering model. Rec- ommendations are then generated quickly by using a simple Nearest Neighbor-based approach. We demonstrate the feasibility of ClustKnn both analytically and empirically. We also show, by comparing with a number of other pop- ular CF algorithms that, apart from being highly scalable and intuitive, ClustKnn provides very good recommenda- tion accuracy as well.