Controllable Multi-Interest Framework for Recommendation

KDD '20: The 26th ACM SIGKDD Conference on Knowledge Discovery and Data Mining Virtual Event CA USA July, 2020, pp. 2942-2951, 2020.

Cited by: 0|Bibtex|Views536|Links
EI
Keywords:
memory networkdeep learningcollaborative filteringideal discounted cumulative gainHit rateMore(18+)
Weibo:
Recommender systems start a new phase owing to the rapid development of deep learning

Abstract:

Recently, neural networks have been widely used in e-commerce recommender systems, owing to the rapid development of deep learning. We formalize the recommender system as a sequential recommendation problem, intending to predict the next items that the user might be interacted with. Recent works usually give an overall embedding from a us...More
0
Introduction
  • The development of e-commerce revolutionized the shopping styles in recent years. Recommender systems play a fundamental role in e-commerce companies.
  • Traditional recommendation methods mainly use collaborative filtering methods [47, 48] to predict scores between users and items.
  • Neural networks have been widely used in e-commerce recommender systems, owing to the rapid development of deep learning.
  • Neural recommender systems generate representations for users and items and outperform traditional recommendation methods.
  • Due to the large-scale e-commerce users and items, it is hard to use deep models to directly give the click-through rate (CTR) prediction between each pair of users and items.
  • Current industrial practice is to use fast K nearest neighbors (e.g., Faiss [25]) to generate the candidate items and use a deep model to integrate the attributes of users and items to optimize the business metrics such as CTR
Highlights
  • The development of e-commerce revolutionized our shopping styles in recent years
  • Neural networks have been widely used in e-commerce recommender systems, owing to the rapid development of deep learning
  • The model performance for the sequential recommendation is shown in Table 3
  • We propose a novel controllable multi-interest framework for the sequential recommendation
  • Recommender systems start a new phase owing to the rapid development of deep learning
  • Traditional recommendation methods cannot meet the requirements of the industry
Methods
  • The authors formulate the problem and introduce the proposed framework in detail, as well as showing the difference between the framework and representative existing methods.

    3.1 Problem Formulation

    Assume the authors have a set of users u ∈ U and a set of items i ∈ I.
  • Et(u) records Notation u i e U the author Ius d K N Vu δ (·).
  • The problem of sequential recommendation is to predict the items that the user might be interacted with.
  • The matching stage corresponds to retrieving top-N candidate items, while the ranking stage is used for sorting the candidate items by more precise scores.
  • In the following parts of this section, the authors will introduce the controllable multi-interest framework and illustrate the significance of the framework for the sequential recommendation problem
Results
  • Each interest of a user independently retrieves top-N candidate items.
  • The authors use the model to compute multiple interests for each user in the test set.
  • The items retrieved by different user interests are fed into the aggregation module.
  • After this module, top-N items out of K · N items are the final candidate items and are used to compute the evaluation metric, recall@50
Conclusion
  • The authors propose a novel controllable multi-interest framework for the sequential recommendation.
  • The authors' framework uses a multi-interest extraction module to generate multiple user interests and uses an aggregation module to obtain the overall top-N items.
  • Experimental results demonstrate that the models can achieve significant improvements over start-of-the-art models on two challenging datasets.
  • Results on the billion-scale industrial dataset further confirm the effectiveness and efficiency of the framework in practice.
  • Traditional recommendation methods cannot meet the requirements of the industry.
  • The authors plan to leverage memory networks to capture the evolving interests of users and introduce cognitive theory to make better user modeling
Summary
  • Introduction:

    The development of e-commerce revolutionized the shopping styles in recent years. Recommender systems play a fundamental role in e-commerce companies.
  • Traditional recommendation methods mainly use collaborative filtering methods [47, 48] to predict scores between users and items.
  • Neural networks have been widely used in e-commerce recommender systems, owing to the rapid development of deep learning.
  • Neural recommender systems generate representations for users and items and outperform traditional recommendation methods.
  • Due to the large-scale e-commerce users and items, it is hard to use deep models to directly give the click-through rate (CTR) prediction between each pair of users and items.
  • Current industrial practice is to use fast K nearest neighbors (e.g., Faiss [25]) to generate the candidate items and use a deep model to integrate the attributes of users and items to optimize the business metrics such as CTR
  • Methods:

    The authors formulate the problem and introduce the proposed framework in detail, as well as showing the difference between the framework and representative existing methods.

    3.1 Problem Formulation

    Assume the authors have a set of users u ∈ U and a set of items i ∈ I.
  • Et(u) records Notation u i e U the author Ius d K N Vu δ (·).
  • The problem of sequential recommendation is to predict the items that the user might be interacted with.
  • The matching stage corresponds to retrieving top-N candidate items, while the ranking stage is used for sorting the candidate items by more precise scores.
  • In the following parts of this section, the authors will introduce the controllable multi-interest framework and illustrate the significance of the framework for the sequential recommendation problem
  • Results:

    Each interest of a user independently retrieves top-N candidate items.
  • The authors use the model to compute multiple interests for each user in the test set.
  • The items retrieved by different user interests are fed into the aggregation module.
  • After this module, top-N items out of K · N items are the final candidate items and are used to compute the evaluation metric, recall@50
  • Conclusion:

    The authors propose a novel controllable multi-interest framework for the sequential recommendation.
  • The authors' framework uses a multi-interest extraction module to generate multiple user interests and uses an aggregation module to obtain the overall top-N items.
  • Experimental results demonstrate that the models can achieve significant improvements over start-of-the-art models on two challenging datasets.
  • Results on the billion-scale industrial dataset further confirm the effectiveness and efficiency of the framework in practice.
  • Traditional recommendation methods cannot meet the requirements of the industry.
  • The authors plan to leverage memory networks to capture the evolving interests of users and introduce cognitive theory to make better user modeling
Tables
  • Table1: Notations
  • Table2: Statistics of datasets
  • Table3: Model performance on public datasets. Bolded numbers are the best performance of each column. All the numbers in the table are percentage numbers with ‘%’ omitted
  • Table4: Model performance of parameter sensitivity. All the numbers in the table are percentage numbers with ‘%’ omitted
  • Table5: Model performance of Amazon dataset for the controllable study. All the numbers in the table are percentage numbers with ‘%’ omitted
  • Table6: Statistics of the industrial dataset
Download tables as Excel
Related work
  • In this section, we introduce the related literature about recommender systems and recommendation diversity, as well as capsule networks and the attention mechanism we used in the paper.

    Collaborative filtering [47, 48] methods have been proven successful in real-world recommender systems, which find similar users and items and make recommendations on this basis. Matrix factorizaion [30] is the most popular technique in classical recommender research, which maps both users and items to a joint latent factor space, such that user-item interactions are modeled as inner products in that space. Factorization Machines (FMs) [44] model all interactions between variables using factorized parameters and thus can estimate interactions even in problems with huge sparsity like recommender systems.

    Neural Recommender Systems. Neural Collaborative Filtering (NCF) [20] uses a neural network architecture to model latent features of users and items. NFM [19] seamlessly combines the linearity of FMs in modeling second-order feature interactions and the non-linearity of neural networks in modeling higher-order feature interactions. DeepFM [14] designs an end-to-end learning model that emphasizes both low-order and high-order feature interactions for CTR prediction. xDeepFM [33] extends DeepFM and can learn specific bounded-degree feature interactions explicitly. Deep Matrix Factorization (DMF) [55] uses a deep structure learning architecture to learn a common low dimensional space for the representations of users and items based on explicit ratings and non-preference implicit feedback. DCN [53] keeps the benefits of a deep model and introduces a novel cross network that is more efficient in learning specific bounded-degree feature interactions. CMN [12] uses deep architecture to unify the two classes of CF models capitalizing on the strengths of the global structure of the latent factor model and local neighborhood-based structure in a nonlinear fashion.
Reference
  • Gediminas Adomavicius and YoungOk Kwon. 201Improving aggregate recommendation diversity using ranking-based techniques. TKDE 24, 5 (2011), 896–911.
    Google ScholarLocate open access versionFindings
  • Sujoy Bag, Abhijeet Ghadge, and Manoj Kumar Tiwari. 2019. An integrated recommender system for improved accuracy and aggregate diversity. Computers & Industrial Engineering 130 (2019), 187–197.
    Google ScholarLocate open access versionFindings
  • Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473 (2014).
    Findings
  • Keith Bradley and Barry Smyth. 2001. Improving recommendation diversity. In AICS’01. Citeseer, 85–94.
    Google ScholarLocate open access versionFindings
  • Peter J Burt. 1988. Attention mechanisms for vision in a dynamic world. In ICPR’88. IEEE, 977–987.
    Google ScholarLocate open access versionFindings
  • Yukuo Cen, Xu Zou, Jianwei Zhang, Hongxia Yang, Jingren Zhou, and Jie Tang. 2019. Representation learning for attributed multiplex heterogeneous network. In KDD’19. 1358–1368.
    Google ScholarFindings
  • Xu Chen, Hongteng Xu, Yongfeng Zhang, Jiaxi Tang, Yixin Cao, Zheng Qin, and Hongyuan Zha. 2018. Sequential recommendation with user memory networks. In WSDM’18. ACM, 108–116.
    Google ScholarLocate open access versionFindings
  • Peizhe Cheng, Shuaiqiang Wang, Jun Ma, Jiankai Sun, and Hui Xiong. 2017. Learning to recommend accurate and diverse items. In WWW’17. 183–192.
    Google ScholarLocate open access versionFindings
  • Paul Covington, Jay Adams, and Emre Sargin. 2016. Deep neural networks for youtube recommendations. In RecSys’16. ACM, 191–198.
    Google ScholarLocate open access versionFindings
  • Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018).
    Findings
  • Tommaso Di Noia, Vito Claudio Ostuni, Jessica Rosati, Paolo Tomeo, and Eugenio Di Sciascio. 2014. An analysis of users’ propensity toward diversity in recommendations. In RecSys’14. 285–288.
    Google ScholarLocate open access versionFindings
  • Travis Ebesu, Bin Shen, and Yi Fang. 2018. Collaborative memory network for recommendation systems. In SIGIR’18. ACM, 515–524.
    Google ScholarLocate open access versionFindings
  • Anupriya Gogna and Angshul Majumdar. 2017. Balancing accuracy and diversity in recommendations using matrix completion framework. Knowledge-Based Systems 125 (2017), 83–95.
    Google ScholarLocate open access versionFindings
  • Huifeng Guo, Ruiming Tang, Yunming Ye, Zhenguo Li, and Xiuqiang He. 2017. DeepFM: a factorization-machine based neural network for CTR prediction. In IJCAI’17.
    Google ScholarFindings
  • Will Hamilton, Zhitao Ying, and Jure Leskovec. 2017. Inductive representation learning on large graphs. In NIPS’17. 1024–1034.
    Google ScholarLocate open access versionFindings
  • Ruining He, Wang-Cheng Kang, and Julian McAuley. 2017. Translation-based recommendation. In RecSys’17. ACM, 161–169.
    Google ScholarLocate open access versionFindings
  • Ruining He and Julian McAuley. 2016. Fusing similarity models with markov chains for sparse sequential recommendation. In ICDM’16. IEEE, 191–200.
    Google ScholarLocate open access versionFindings
  • Ruining He and Julian McAuley. 2016. Ups and downs: Modeling the visual evolution of fashion trends with one-class collaborative filtering. In WWW’16. International World Wide Web Conferences Steering Committee, 507–517.
    Google ScholarLocate open access versionFindings
  • Xiangnan He and Tat-Seng Chua. 2017. Neural factorization machines for sparse predictive analytics. In SIGIR’17. ACM, 355–364.
    Google ScholarLocate open access versionFindings
  • Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. 2017. Neural collaborative filtering. In WWW’17. International World Wide Web Conferences Steering Committee, 173–182.
    Google ScholarLocate open access versionFindings
  • Balázs Hidasi, Alexandros Karatzoglou, Linas Baltrunas, and Domonkos Tikk. 2016. Session-based recommendations with recurrent neural networks. In ICLR’16.
    Google ScholarFindings
  • Geoffrey E Hinton, Alex Krizhevsky, and Sida D Wang. 2011. Transforming auto-encoders. In ICANN’11. Springer, 44–51.
    Google ScholarFindings
  • Kalervo Järvelin and Jaana Kekäläinen. 2000. IR evaluation methods for retrieving highly relevant documents. In SIGIR’00. ACM, 41–48.
    Google ScholarLocate open access versionFindings
  • Sébastien Jean, Kyunghyun Cho, Roland Memisevic, and Yoshua Bengio. 2015. On using very large target vocabulary for neural machine translation. ACL’15.
    Google ScholarFindings
  • Jeff Johnson, Matthijs Douze, and Hervé Jégou. 2017. Billion-scale similarity search with GPUs. arXiv preprint arXiv:1702.08734 (2017).
    Findings
  • M Kalaivanan and K Vengatesan. 2013. Recommendation system based on statistical analysis of ranking from user. In ICICES’13. IEEE, 479–484.
    Google ScholarLocate open access versionFindings
  • Wang-Cheng Kang and Julian McAuley. 2018. Self-attentive sequential recommendation. In ICDM’18. IEEE, 197–206.
    Google ScholarLocate open access versionFindings
  • George Karypis. 2001. Evaluation of item-based top-n recommendation algorithms. In CIKM’01. ACM, 247–254.
    Google ScholarLocate open access versionFindings
  • Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).
    Findings
  • Yehuda Koren, Robert Bell, and Chris Volinsky. 2009. Matrix factorization techniques for recommender systems. Computer 8 (2009), 30–37.
    Google ScholarLocate open access versionFindings
  • Chao Li, Zhiyuan Liu, Mengmeng Wu, Yuchi Xu, Pipei Huang, Huan Zhao, Guoliang Kang, Qiwei Chen, Wei Li, and Dik Lun Lee. 2019. Multi-Interest Network with Dynamic Routing for Recommendation at Tmall. arXiv preprint arXiv:1904.08030 (2019).
    Findings
  • Chenliang Li, Cong Quan, Li Peng, Yunwei Qi, Yuming Deng, and Libing Wu. 2019. A Capsule Network for Recommendation and Explaining What You Like and Dislike. In SIGIR’19. ACM, 275–284.
    Google ScholarLocate open access versionFindings
  • Jianxun Lian, Xiaohuan Zhou, Fuzheng Zhang, Zhongxia Chen, Xing Xie, and Guangzhong Sun. 2018. xDeepFM: Combining explicit and implicit feature interactions for recommender systems. In KDD’18. ACM, 1754–1763.
    Google ScholarLocate open access versionFindings
  • Dawen Liang, Rahul G Krishnan, Matthew D Hoffman, and Tony Jebara. 2018. Variational autoencoders for collaborative filtering. In WWW’18. 689–698.
    Google ScholarLocate open access versionFindings
  • Zhouhan Lin, Minwei Feng, Cicero Nogueira dos Santos, Mo Yu, Bing Xiang, Bowen Zhou, and Yoshua Bengio. 2017. A structured self-attentive sentence embedding. In ICLR’17.
    Google ScholarFindings
  • Fuyu Lv, Taiwei Jin, Changlong Yu, Fei Sun, Quan Lin, Keping Yang, and Wilfred Ng. 2019. SDM: Sequential deep matching model for online large-scale recommender system. In CIKM’19. 2635–2643.
    Google ScholarFindings
  • Jianxin Ma, Chang Zhou, Peng Cui, Hongxia Yang, and Wenwu Zhu. 2019. Learning disentangled representations for recommendation. In NIPS’19. 5712–5723.
    Google ScholarFindings
  • Benjamin Marlin. 2004. Collaborative filtering: A machine learning perspective. University of Toronto Toronto.
    Google ScholarFindings
  • Julian McAuley, Christopher Targett, Qinfeng Shi, and Anton Van Den Hengel. 2015. Image-based recommendations on styles and substitutes. In SIGIR’15. ACM, 43–52.
    Google ScholarLocate open access versionFindings
  • Katja Niemann and Martin Wolpers. 2013. A new collaborative filtering approach for increasing the aggregate diversity of recommender systems. In KDD’13. 955– 963.
    Google ScholarLocate open access versionFindings
  • Umberto Panniello, Alexander Tuzhilin, and Michele Gorgoglione. 2014. Comparing context-aware recommender systems in terms of accuracy and diversity. UMUAI 24, 1-2 (2014), 35–65.
    Google ScholarLocate open access versionFindings
  • Qi Pi, Weijie Bian, Guorui Zhou, Xiaoqiang Zhu, and Kun Gai. 2019. Practice on long sequential user behavior modeling for click-through rate prediction. In KDD’19. 2671–2679.
    Google ScholarFindings
  • Lijing Qin and Xiaoyan Zhu. 2013. Promoting diversity in recommendation by entropy regularizer. In IJCAI’13.
    Google ScholarFindings
  • Steffen Rendle. 2010. Factorization machines. In ICDM’10. IEEE, 995–1000.
    Google ScholarLocate open access versionFindings
  • Steffen Rendle, Christoph Freudenthaler, and Lars Schmidt-Thieme. 2010. Factorizing personalized markov chains for next-basket recommendation. In WWW’10. ACM, 811–820.
    Google ScholarLocate open access versionFindings
  • Sara Sabour, Nicholas Frosst, and Geoffrey E Hinton. 2017. Dynamic routing between capsules. In NIPS’17. 3856–3866.
    Google ScholarLocate open access versionFindings
  • Badrul Munir Sarwar, George Karypis, Joseph A Konstan, John Riedl, et al. 2001. Item-based collaborative filtering recommendation algorithms. WWW’01 (2001), 285–295.
    Google ScholarFindings
  • J Ben Schafer, Dan Frankowski, Jon Herlocker, and Shilad Sen. 2007. Collaborative filtering recommender systems. In The adaptive web. Springer, 291–324.
    Google ScholarFindings
  • Malcolm Slaney and William White. 2006. Measuring playlist diversity for recommendation systems. In AMCMM’06 workshop. 77–82.
    Google ScholarLocate open access versionFindings
  • Yaoru Sun and Robert Fisher. 2003. Object-based visual attention for computer vision. Artificial intelligence 146, 1 (2003), 77–123.
    Google ScholarFindings
  • Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In NIPS’17. 5998–6008.
    Google ScholarLocate open access versionFindings
  • Pengfei Wang, Jiafeng Guo, Yanyan Lan, Jun Xu, Shengxian Wan, and Xueqi Cheng. 2015. Learning hierarchical representation model for nextbasket recommendation. In SIGIR’15. ACM, 403–412.
    Google ScholarLocate open access versionFindings
  • Ruoxi Wang, Bin Fu, Gang Fu, and Mingliang Wang. 2017.
    Google ScholarFindings
  • Chao-Yuan Wu, Amr Ahmed, Alex Beutel, Alexander J Smola, and How Jing. 2017. Recurrent recommender networks. In WSDM’17. ACM, 495–503.
    Google ScholarLocate open access versionFindings
  • Hong-Jian Xue, Xinyu Dai, Jianbing Zhang, Shujian Huang, and Jiajun Chen. 2017. Deep Matrix Factorization Models for Recommender Systems.. In IJCAI’17. 3203–3209.
    Google ScholarLocate open access versionFindings
  • Rex Ying, Ruining He, Kaifeng Chen, Pong Eksombatchai, William L Hamilton, and Jure Leskovec. 2018. Graph convolutional neural networks for web-scale recommender systems. In KDD’18. ACM, 974–983.
    Google ScholarLocate open access versionFindings
  • Feng Yu, Qiang Liu, Shu Wu, Liang Wang, and Tieniu Tan. 2016. A dynamic recurrent model for next basket recommendation. In SIGIR’16. ACM, 729–732.
    Google ScholarLocate open access versionFindings
  • Ting Yu, Junpeng Guo, Wenhua Li, Harry Jiannan Wang, and Ling Fan. 2019. Recommendation with diversity: An adaptive trust-aware model. Decision Support Systems 123 (2019), 113073.
    Google ScholarLocate open access versionFindings
  • Chang Zhou, Jinze Bai, Junshuai Song, Xiaofei Liu, Zhengchao Zhao, Xiusi Chen, and Jun Gao. 2018. Atrank: An attention-based user behavior modeling framework for recommendation. In AAAI’18.
    Google ScholarFindings
  • Guorui Zhou, Xiaoqiang Zhu, Chenru Song, Ying Fan, Han Zhu, Xiao Ma, Yanghui Yan, Junqi Jin, Han Li, and Kun Gai. 2018. Deep interest network for click-through rate prediction. In KDD’18. ACM, 1059–1068.
    Google ScholarLocate open access versionFindings
  • Han Zhu, Xiang Li, Pengye Zhang, Guozheng Li, Jie He, Han Li, and Kun Gai. 2018. Learning tree-based deep model for recommender systems. In KDD’18. ACM, 1079–1088. 5 https://www.tensorflow.org/6 https://data.aliyun.com/7 https://github.com/facebookresearch/faiss 8 https://github.com/cenyk1230/ComiRec
    Findings
  • 9 http://jmcauley.ucsd.edu/data/amazon/10 https://tianchi.aliyun.com/dataset/dataDetail?dataId=649&userId=1
    Findings
Your rating :
0

 

Tags
Comments