Neural Collaborative Filtering

    WWW, pp. 173-182, 2017.

    Cited by: 1141|Bibtex|Views47|Links
    EI
    Keywords:
    collaborative denoising autoencoderimplicit coordinate descentnatural languagemulti-layer perceptroninteraction functionMore(18+)
    Wei bo:
    The performance of a ranked list is judged by Hit Ratio and Normalized Discounted Cumulative Gain

    Abstract:

    In recent years, deep neural networks have yielded immense success on speech recognition, computer vision and natural language processing. However, the exploration of deep neural networks on recommender systems has received relatively less scrutiny. In this work, we strive to develop techniques based on neural networks to tackle the key p...More

    Code:

    Data:

    Introduction
    • In the era of information explosion, recommender systems play a pivotal role in alleviating information overload, having been widely adopted by many online services, including E-commerce, online news and social media sites.
    • For the task of rating prediction on explicit feedback, it is well known that the performance of the MF model can be improved by incorporating user and item bias terms into the interaction function1
    • While it seems to be just a trivial tweak for the inner product operator [14], it points to the positive effect of designing a better, dedicated interaction function for modelling the latent feature interactions between users and items.
    • The inner product, which combines the multiplication of latent features linearly, may not be sufficient to capture the complex structure of user interaction data
    Highlights
    • In the era of information explosion, recommender systems play a pivotal role in alleviating information overload, having been widely adopted by many online services, including E-commerce, online news and social media sites
    • This paper explores the use of deep neural networks for learning the interaction function from data, rather than a handcraft that has been done by many previous work [18, 21]
    • We present a neural network architecture to model latent features of users and items and devise a general framework NCF for collaborative filtering based on neural networks
    • We show that matrix factorization can be interpreted as a specialization of NCF and utilize a multi-layer perceptron to endow NCF modelling with a high level of non-linearities
    • The performance of a ranked list is judged by Hit Ratio (HR) and Normalized Discounted Cumulative Gain (NDCG) [11]
    • The Hit Ratio intuitively measures whether the test item is present on the top-10 list, and the Normalized Discounted Cumulative Gain accounts for the position of the hit by assigning higher scores to hits at top ranks
    Methods
    • (12) where pGu and pM u denote the user embedding for GMF and MLP parts, respectively; and similar notations of qGi and qM i for item embeddings.
    • The authors use ReLU as the activation function of MLP layers.
    • This model combines the linearity of MF and non-linearity of DNNs for modelling user–item latent structures.
    Results
    • Evaluation Protocols

      To evaluate the performance of item recommendation, the authors adopted the leave-one-out evaluation, which has been widely used in literature [1, 14, 27].
    • The HR intuitively measures whether the test item is present on the top-10 list, and the NDCG accounts for the position of the hit by assigning higher scores to hits at top ranks.
    • The authors calculated both metrics for each test user and reported the average score
    Conclusion
    • CONCLUSION AND FUTURE

      WORK

      In this work, the authors explored neural network architectures for collaborative filtering.
    • To build a multi-media recommender system, the authors need to develop effective methods to learn from multi-view and multi-modal data [13, 40].
    • Another emerging direction is to explore the potential of recurrent neural networks and hashing methods [46] for providing efficient online recommendation [14, 1]
    Summary
    • Introduction:

      In the era of information explosion, recommender systems play a pivotal role in alleviating information overload, having been widely adopted by many online services, including E-commerce, online news and social media sites.
    • For the task of rating prediction on explicit feedback, it is well known that the performance of the MF model can be improved by incorporating user and item bias terms into the interaction function1
    • While it seems to be just a trivial tweak for the inner product operator [14], it points to the positive effect of designing a better, dedicated interaction function for modelling the latent feature interactions between users and items.
    • The inner product, which combines the multiplication of latent features linearly, may not be sufficient to capture the complex structure of user interaction data
    • Methods:

      (12) where pGu and pM u denote the user embedding for GMF and MLP parts, respectively; and similar notations of qGi and qM i for item embeddings.
    • The authors use ReLU as the activation function of MLP layers.
    • This model combines the linearity of MF and non-linearity of DNNs for modelling user–item latent structures.
    • Results:

      Evaluation Protocols

      To evaluate the performance of item recommendation, the authors adopted the leave-one-out evaluation, which has been widely used in literature [1, 14, 27].
    • The HR intuitively measures whether the test item is present on the top-10 list, and the NDCG accounts for the position of the hit by assigning higher scores to hits at top ranks.
    • The authors calculated both metrics for each test user and reported the average score
    • Conclusion:

      CONCLUSION AND FUTURE

      WORK

      In this work, the authors explored neural network architectures for collaborative filtering.
    • To build a multi-media recommender system, the authors need to develop effective methods to learn from multi-view and multi-modal data [13, 40].
    • Another emerging direction is to explore the potential of recurrent neural networks and hashing methods [46] for providing efficient online recommendation [14, 1]
    Tables
    • Table1: Statistics of the evaluation datasets
    • Table2: Performance of NeuMF with and without pre-training
    • Table3: HR@10 of MLP with different layers
    • Table4: NDCG@10 of MLP with different layers
    Download tables as Excel
    Related work
    • While early literature on recommendation has largely focused on explicit feedback [30, 31], recent attention is increasingly shifting towards implicit data [1, 14, 23]. The collaborative filtering (CF) task with implicit feedback is usually formulated as an item recommendation problem, for which the aim is to recommend a short list of items to users. In contrast to rating prediction that has been widely solved by work on explicit feedback, addressing the item recommendation problem is more practical but challenging [1, 11]. One key insight is to model the missing data, which are always ignored by the work on explicit feedback [21, 48]. To tailor latent factor models for item recommendation with implicit feedback, early work [19, 27] applies a uniform weighting where two strategies have been proposed — which either treated all missing data as negative instances [19] or sampled negative instances from missing data [27]. Recently, He et al [14] and Liang et al [23] proposed dedicated models to weight missing data, and Rendle et al [1] developed an
    Funding
    • ∗NExT research is supported by the National Research Foundation, Prime Minister’s Office, Singapore under its IRC@SG Funding Initiative
    Reference
    • I. Bayer, X. He, B. Kanagal, and S. Rendle. A generic coordinate descent framework for learning from implicit feedback. In WWW, 2017.
      Google ScholarLocate open access versionFindings
    • A. Bordes, N. Usunier, A. Garcia-Duran, J. Weston, and O. Yakhnenko. Translating embeddings for modeling multi-relational data. In NIPS, pages 2787–2795, 2013.
      Google ScholarLocate open access versionFindings
    • T. Chen, X. He, and M.-Y. Kan. Context-aware image tweet modelling and recommendation. In MM, pages 1018–1027, 2016.
      Google ScholarLocate open access versionFindings
    • H.-T. Cheng, L. Koc, J. Harmsen, T. Shaked, T. Chandra, H. Aradhye, G. Anderson, G. Corrado, W. Chai, M. Ispir, et al. Wide & deep learning for recommender systems. arXiv preprint arXiv:1606.07792, 2016.
      Findings
    • R. Collobert and J. Weston. A unified architecture for natural language processing: Deep neural networks with multitask learning. In ICML, pages 160–167, 2008.
      Google ScholarLocate open access versionFindings
    • A. M. Elkahky, Y. Song, and X. He. A multi-view deep learning approach for cross domain user modeling in recommendation systems. In WWW, pages 278–288, 2015.
      Google ScholarLocate open access versionFindings
    • D. Erhan, Y. Bengio, A. Courville, P.-A. Manzagol, P. Vincent, and S. Bengio. Why does unsupervised pre-training help deep learning? Journal of Machine Learning Research, 11:625–660, 2010.
      Google ScholarLocate open access versionFindings
    • X. Geng, H. Zhang, J. Bian, and T.-S. Chua. Learning image and user features for recommendation in social networks. In ICCV, pages 4274–4282, 2015.
      Google ScholarLocate open access versionFindings
    • X. Glorot, A. Bordes, and Y. Bengio. Deep sparse rectifier neural networks. In AISTATS, pages 315–323, 2011.
      Google ScholarLocate open access versionFindings
    • K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In CVPR, 2016.
      Google ScholarLocate open access versionFindings
    • X. He, T. Chen, M.-Y. Kan, and X. Chen. TriRank: Review-aware explainable recommendation by modeling aspects. In CIKM, pages 1661–1670, 2015.
      Google ScholarLocate open access versionFindings
    • X. He, M. Gao, M.-Y. Kan, Y. Liu, and K. Sugiyama. Predicting the popularity of web 2.0 items based on user comments. In SIGIR, pages 233–242, 2014.
      Google ScholarLocate open access versionFindings
    • X. He, M.-Y. Kan, P. Xie, and X. Chen. Comment-based multi-view clustering of web 2.0 items. In WWW, pages 771–782, 2014.
      Google ScholarLocate open access versionFindings
    • X. He, H. Zhang, M.-Y. Kan, and T.-S. Chua. Fast matrix factorization for online recommendation with implicit feedback. In SIGIR, pages 549–558, 2016.
      Google ScholarLocate open access versionFindings
    • R. Hong, Z. Hu, L. Liu, M. Wang, S. Yan, and Q. Tian. Understanding blooming human groups in social networks. IEEE Transactions on Multimedia, 17(11):1980–1988, 2015.
      Google ScholarLocate open access versionFindings
    • R. Hong, Y. Yang, M. Wang, and X. S. Hua. Learning visual semantic relationships for efficient visual retrieval. IEEE Transactions on Big Data, 1(4):152–161, 2015.
      Google ScholarLocate open access versionFindings
    • K. Hornik, M. Stinchcombe, and H. White. Multilayer feedforward networks are universal approximators. Neural Networks, 2(5):359–366, 1989.
      Google ScholarLocate open access versionFindings
    • L. Hu, A. Sun, and Y. Liu. Your neighbors affect your ratings: On geographical neighborhood influence to rating prediction. In SIGIR, pages 345–354, 2014.
      Google ScholarLocate open access versionFindings
    • Y. Hu, Y. Koren, and C. Volinsky. Collaborative filtering for implicit feedback datasets. In ICDM, pages 263–272, 2008.
      Google ScholarLocate open access versionFindings
    • D. Kingma and J. Ba. Adam: A method for stochastic optimization. In ICLR, pages 1–15, 2014.
      Google ScholarLocate open access versionFindings
    • Y. Koren. Factorization meets the neighborhood: A multifaceted collaborative filtering model. In KDD, pages 426–434, 2008.
      Google ScholarLocate open access versionFindings
    • S. Li, J. Kawale, and Y. Fu. Deep collaborative filtering via marginalized denoising auto-encoder. In CIKM, pages 811–820, 2015.
      Google ScholarLocate open access versionFindings
    • D. Liang, L. Charlin, J. McInerney, and D. M. Blei. Modeling user exposure in recommendation. In WWW, pages 951–961, 2016.
      Google ScholarLocate open access versionFindings
    • M. Nickel, K. Murphy, V. Tresp, and E. Gabrilovich. A review of relational machine learning for knowledge graphs. Proceedings of the IEEE, 104:11–33, 2016.
      Google ScholarLocate open access versionFindings
    • X. Ning and G. Karypis. Slim: Sparse linear methods for top-n recommender systems. In ICDM, pages 497–506, 2011.
      Google ScholarLocate open access versionFindings
    • S. Rendle. Factorization machines. In ICDM, pages 995–1000, 2010.
      Google ScholarLocate open access versionFindings
    • S. Rendle, C. Freudenthaler, Z. Gantner, and L. Schmidt-Thieme. Bpr: Bayesian personalized ranking from implicit feedback. In UAI, pages 452–461, 2009.
      Google ScholarLocate open access versionFindings
    • S. Rendle, Z. Gantner, C. Freudenthaler, and L. Schmidt-Thieme. Fast context-aware recommendations with factorization machines. In SIGIR, pages 635–644, 2011.
      Google ScholarLocate open access versionFindings
    • R. Salakhutdinov and A. Mnih. Probabilistic matrix factorization. In NIPS, pages 1–8, 2008.
      Google ScholarLocate open access versionFindings
    • R. Salakhutdinov, A. Mnih, and G. Hinton. Restricted boltzmann machines for collaborative filtering. In ICDM, pages 791–798, 2007.
      Google ScholarLocate open access versionFindings
    • B. Sarwar, G. Karypis, J. Konstan, and J. Riedl. Item-based collaborative filtering recommendation algorithms. In WWW, pages 285–295, 2001.
      Google ScholarLocate open access versionFindings
    • S. Sedhain, A. K. Menon, S. Sanner, and L. Xie. Autorec: Autoencoders meet collaborative filtering. In WWW, pages 111–112, 2015.
      Google ScholarLocate open access versionFindings
    • R. Socher, D. Chen, C. D. Manning, and A. Ng. Reasoning with neural tensor networks for knowledge base completion. In NIPS, pages 926–934, 2013.
      Google ScholarLocate open access versionFindings
    • N. Srivastava and R. R. Salakhutdinov. Multimodal learning with deep boltzmann machines. In NIPS, pages 2222–2230, 2012.
      Google ScholarLocate open access versionFindings
    • F. Strub and J. Mary. Collaborative filtering with stacked denoising autoencoders and sparse inputs. In NIPS Workshop on Machine Learning for eCommerce, 2015.
      Google ScholarLocate open access versionFindings
    • T. T. Truyen, D. Q. Phung, and S. Venkatesh. Ordinal boltzmann machines for collaborative filtering. In UAI, pages 548–556, 2009.
      Google ScholarLocate open access versionFindings
    • A. Van den Oord, S. Dieleman, and B. Schrauwen. Deep content-based music recommendation. In NIPS, pages 2643–2651, 2013.
      Google ScholarLocate open access versionFindings
    • H. Wang, N. Wang, and D.-Y. Yeung. Collaborative deep learning for recommender systems. In KDD, pages 1235–1244, 2015.
      Google ScholarLocate open access versionFindings
    • M. Wang, W. Fu, S. Hao, D. Tao, and X. Wu. Scalable semi-supervised learning by efficient anchor graph regularization. IEEE Transactions on Knowledge and Data Engineering, 28(7):1864–1877, 2016.
      Google ScholarLocate open access versionFindings
    • M. Wang, H. Li, D. Tao, K. Lu, and X. Wu. Multimodal graph-based reranking for web image search. IEEE Transactions on Image Processing, 21(11):4649–4661, 2012.
      Google ScholarLocate open access versionFindings
    • M. Wang, X. Liu, and X. Wu. Visual classification by l1 hypergraph modeling. IEEE Transactions on Knowledge and Data Engineering, 27(9):2564–2574, 2015.
      Google ScholarLocate open access versionFindings
    • X. Wang, L. Nie, X. Song, D. Zhang, and T.-S. Chua. Unifying virtual and physical worlds: Learning towards local and global consistency. ACM Transactions on Information Systems, 2017.
      Google ScholarLocate open access versionFindings
    • X. Wang and Y. Wang. Improving content-based and hybrid music recommendation using deep learning. In MM, pages 627–636, 2014.
      Google ScholarLocate open access versionFindings
    • Y. Wu, C. DuBois, A. X. Zheng, and M. Ester. Collaborative denoising auto-encoders for top-n recommender systems. In WSDM, pages 153–162, 2016.
      Google ScholarLocate open access versionFindings
    • F. Zhang, N. J. Yuan, D. Lian, X. Xie, and W.-Y. Ma. Collaborative knowledge base embedding for recommender systems. In KDD, pages 353–362, 2016.
      Google ScholarLocate open access versionFindings
    • H. Zhang, F. Shen, W. Liu, X. He, H. Luan, and T.-S. Chua. Discrete collaborative filtering. In SIGIR, pages 325–334, 2016.
      Google ScholarLocate open access versionFindings
    • H. Zhang, Y. Yang, H. Luan, S. Yang, and T.-S. Chua. Start from scratch: Towards automatically identifying, modeling, and naming visual attributes. In MM, pages 187–196, 2014.
      Google ScholarLocate open access versionFindings
    • Y. Zheng, B. Tang, W. Ding, and H. Zhou. A neural autoregressive approach to collaborative filtering. In ICML, pages 764–773, 2016.
      Google ScholarLocate open access versionFindings
    Your rating :
    0

     

    Tags
    Comments