Sequential Scenario-Specific Meta Learner for Online Recommendation

KDD, 2019.

Cited by: 8|Bibtex|Views153|Links
EI
Keywords:
few-shot learning meta learning neural networks personalized ranking recommender systems
Weibo:
This paper addresses such problems using few-shot learning and meta learning

Abstract:

Cold-start problems are long-standing challenges for practical recommendations. Most existing recommendation algorithms rely on extensive observed data and are brittle to recommendation scenarios with few interactions. This paper addresses such problems usingfew-shot learning andmeta learning. Our approach is based on the insight that hav...More

Code:

Data:

0
Introduction
  • The personalized recommendation is an important method for information retrieval and content discovery in today’s information-rich environment.
  • The mapping result can be a real value for explicit ratings or a binary value for implicit feedback [18, 26, 26, 47]
  • This setting usually assumes that the behavior pattern of the same user is relatively stationary in different contexts, which is not true in many practical tasks [31, 36].
  • It has been shown that including contextual information leads to better predictive models and better quality of recommendations [1, 35]
Highlights
  • The personalized recommendation is an important method for information retrieval and content discovery in today’s information-rich environment
  • This paper addresses the problem of cold-start scenario recommendation with the recent progress on few-shot learning [44, 52, 55] and meta-learning [4, 14, 34, 39]
  • Notice that, compared to previous works in cross-domain recommender systems that require a large amount of training samples on each domain, our work studies the case where the size of Dctrain is limited
  • We explored few-shot learning for recommendation in the scenario-specific setting
  • We proposed a novel sequential scenario-specific framework for recommender systems using meta learning, to solve the cold-start problem in some recommendation scenarios. s2Meta can automatically control the learning process to converge to a good solution and avoid overfitting, which has been the critical issue of few-shot learning
Methods
  • ItemPop CDCF CMF EMCDR CoNet s2Meta Improve Recall@10 Amazon Recall@20 Recall@50.
  • Movielens Recall@10 Recall@20 Recall@50 Recall@20 Taobao Recall@50 Recall@100.
  • 1.90 user interests relatively concentrate in a specific scenario.
  • The authors can find that the deep cross-domain methods can outperform shallow cross-domain methods by improvements over 10%.
  • The performance of NeuMF, which is not designed for cross-domain recommendation, is superior to CMF and CDCF, showing the power of deep learning in recommendation
Results
  • The authors can see that the complete model can significantly outperform the three weakened variations, indicating that three parts of the meta learner all help to improve the final performance.
Conclusion
  • The authors explored few-shot learning for recommendation in the scenario-specific setting.
  • The authors proposed a novel sequential scenario-specific framework for recommender systems using meta learning, to solve the cold-start problem in some recommendation scenarios.
  • Experiments on real-world datasets demonstrate the effectiveness of the proposed method, by comparing with shallow/deep, general/scenario-specific baselines.
  • The authors found that the performance of the same architecture might differ in different datasets.
  • By learning to choose the optimal recommender architecture, the performance of s2Meta can be further improved
Summary
  • Introduction:

    The personalized recommendation is an important method for information retrieval and content discovery in today’s information-rich environment.
  • The mapping result can be a real value for explicit ratings or a binary value for implicit feedback [18, 26, 26, 47]
  • This setting usually assumes that the behavior pattern of the same user is relatively stationary in different contexts, which is not true in many practical tasks [31, 36].
  • It has been shown that including contextual information leads to better predictive models and better quality of recommendations [1, 35]
  • Methods:

    ItemPop CDCF CMF EMCDR CoNet s2Meta Improve Recall@10 Amazon Recall@20 Recall@50.
  • Movielens Recall@10 Recall@20 Recall@50 Recall@20 Taobao Recall@50 Recall@100.
  • 1.90 user interests relatively concentrate in a specific scenario.
  • The authors can find that the deep cross-domain methods can outperform shallow cross-domain methods by improvements over 10%.
  • The performance of NeuMF, which is not designed for cross-domain recommendation, is superior to CMF and CDCF, showing the power of deep learning in recommendation
  • Results:

    The authors can see that the complete model can significantly outperform the three weakened variations, indicating that three parts of the meta learner all help to improve the final performance.
  • Conclusion:

    The authors explored few-shot learning for recommendation in the scenario-specific setting.
  • The authors proposed a novel sequential scenario-specific framework for recommender systems using meta learning, to solve the cold-start problem in some recommendation scenarios.
  • Experiments on real-world datasets demonstrate the effectiveness of the proposed method, by comparing with shallow/deep, general/scenario-specific baselines.
  • The authors found that the performance of the same architecture might differ in different datasets.
  • By learning to choose the optimal recommender architecture, the performance of s2Meta can be further improved
Tables
  • Table1: Notations
  • Table2: Statistics of the Datasets. #Inter. denotes the number of user-item interactions and #Scen. denotes the number of scenarios we use as few-shot tasks
  • Table3: The top-N recall results on test scenarios
  • Table4: Impact of different parts in meta learner on Amazon dataset
  • Table5: Dataset split on Amazon
Related work
  • In this section, we go over the related works on context-aware recommendation, cross-domain recommendation, and meta learning respectively.

    2.1 Context-Aware Recommendation

    The importance of contextual information has been recognized by researchers and practitioners in many disciplines, including ECommerce personalization [35], information retrieval [23], data mining [6] and marketing [36], among many others [31, 46]. Relevant contextual information does matter in recommender systems, and it is important to take account of contextual information, such as time, location, or acquaintances’ impacts [1, 2, 35]. Compared to traditional recommender systems that make predictions based on the information of users and items, context-aware recommender systems [1, 3] make predictions in the space of U × I × C, where C is the context space. The contextual information can be observable (e.g., time, location, etc), or unobservable (e.g., users’ intention). The latter case is related to Session-based Recommendation [19] or Sequence-aware Recommendation [38]. Even if the contextual information is fully-observable, the weight of different types of information is entirely domain-dependent and quite tricky to be tuned for cold-start scenarios.
Funding
  • The work is supported by the NSFC for Distinguished Young Scholar (61825602), Tsinghua University Initiative Scientific Research Program, and a research fund supported by Alibaba
Reference
  • Gediminas Adomavicius, Ramesh Sankaranarayanan, Shahana Sen, and Alexander Tuzhilin. 2005. Incorporating contextual information in recommender systems using a multidimensional approach. ACM Trans. Inf. Syst. 23, 1 (2005), 103–145.
    Google ScholarLocate open access versionFindings
  • Gediminas Adomavicius and Alexander Tuzhilin. 2005. Toward the Next Generation of Recommender Systems: A Survey of the State-of-the-Art and Possible Extensions. IEEE Trans. Knowl. Data Eng. 17, 6 (2005), 734–749.
    Google ScholarLocate open access versionFindings
  • Gediminas Adomavicius and Alexander Tuzhilin. 2015. Context-Aware Recommender Systems. In Recommender Systems Handbook. 191–226.
    Google ScholarLocate open access versionFindings
  • Marcin Andrychowicz, Misha Denil, Sergio Gomez Colmenarejo, Matthew W. Hoffman, David Pfau, Tom Schaul, and Nando de Freitas. 2016. Learning to learn by gradient descent by gradient descent. In NIPS. 3981–3989.
    Google ScholarFindings
  • Parivash Ashrafi, Yi Sun, Neil Davey, Rod Adams, Marc B. Brown, Maria Prapopoulou, and Gary P. Moss. 201The importance of hyperparameters selection within small datasets. In IJCNN. 1–8.
    Google ScholarLocate open access versionFindings
  • Michael J. A. Berry and Gordon Linoff. 1997. Data mining techniques - for marketing, sales, and customer support. Wiley.
    Google ScholarFindings
  • Pavel Brazdil, Christophe Giraud-Carrier, Carlos Soares, and Ricardo Vilalta. 2008. Metalearning: Applications to Data Mining (1 ed.). Springer Publishing Company, Incorporated.
    Google ScholarFindings
  • Iván Cantador, Ignacio Fernández-Tobías, Shlomo Berkovsky, and Paolo Cremonesi. 2015. Cross-Domain Recommender Systems. In Recommender Systems Handbook. 919–959.
    Google ScholarLocate open access versionFindings
  • Heng-Tze Cheng, Levent Koc, Jeremiah Harmsen, Tal Shaked, Tushar Chandra, Hrishi Aradhye, Glen Anderson, Greg Corrado, Wei Chai, Mustafa Ispir, Rohan Anil, Zakaria Haque, Lichan Hong, Vihan Jain, Xiaobing Liu, and Hemal Shah. 2016.
    Google ScholarFindings
  • Paul Covington, Jay Adams, and Emre Sargin. 2016. Deep Neural Networks for YouTube Recommendations. In RecSys. 191–198.
    Google ScholarFindings
  • Tiago Cunha, Carlos Soares, and Andr C.P.L.F. de Carvalho. 2018. Metalearning and Recommender Systems. Inf. Sci. 423, C (Jan. 2018), 128–144.
    Google ScholarLocate open access versionFindings
  • Tiago Cunha, Carlos Soares, and André C. P. L. F. de Carvalho. 2016. Selecting Collaborative Filtering Algorithms Using Metalearning. In ECML/PKDD. 393– 409.
    Google ScholarFindings
  • Michael D. Ekstrand and John Riedl. 2012. When recommenders fail: predicting recommender failure for algorithm selection and combination. In RecSys. 233– 236.
    Google ScholarFindings
  • Chelsea Finn, Pieter Abbeel, and Sergey Levine. 2017. Model-Agnostic MetaLearning for Fast Adaptation of Deep Networks. In ICML. 1126–1135.
    Google ScholarFindings
  • William L. Hamilton, Zhitao Ying, and Jure Leskovec. 2017. Inductive Representation Learning on Large Graphs. In NIPS. 1025–1035.
    Google ScholarFindings
  • F. Maxwell Harper and Joseph A. Konstan. 20The MovieLens Datasets: History and Context. TiiS 5, 4 (2016), 19:1–19:19.
    Google ScholarLocate open access versionFindings
  • Ruining He and Julian McAuley. 2016. Ups and Downs: Modeling the Visual Evolution of Fashion Trends with One-Class Collaborative Filtering. In WWW. 507–517.
    Google ScholarLocate open access versionFindings
  • Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. 2017. Neural Collaborative Filtering. In WWW. 173–182.
    Google ScholarLocate open access versionFindings
  • Balázs Hidasi and Alexandros Karatzoglou. 2018. Recurrent Neural Networks with Top-k Gains for Session-based Recommendations. In CIKM. 843–852.
    Google ScholarLocate open access versionFindings
  • Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long Short-Term Memory. Neural Computation 9, 8 (1997), 1735–1780.
    Google ScholarLocate open access versionFindings
  • Guangneng Hu, Yu Zhang, and Qiang Yang. 2018. CoNet: Collaborative Cross Networks for Cross-Domain Recommendation. In CIKM. 667–676.
    Google ScholarLocate open access versionFindings
  • Sergio Ilarri, Raquel Trillo Lado, and Ramón Hermoso. 2018. Datasets for ContextAware Recommender Systems: Current Context and Possible Directions. In ICDE Workshops. 25–28.
    Google ScholarLocate open access versionFindings
  • Peter Ingwersen and Kalervo Järvelin. 2005. Information Retrieval in Context: IRiX. SIGIR Forum 39, 2 (Dec. 2005), 31–39.
    Google ScholarLocate open access versionFindings
  • Diederick P Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In International Conference on Learning Representations (ICLR).
    Google ScholarLocate open access versionFindings
  • Gregory Koch, Richard Zemel, and Ruslan Salakhutdinov. 2015. Siamese neural networks for one-shot image recognition. In ICML Deep Learning Workshop, Vol. 2.
    Google ScholarLocate open access versionFindings
  • Yehuda Koren, Robert M. Bell, and Chris Volinsky. 2009. Matrix Factorization Techniques for Recommender Systems. IEEE Computer 42, 8 (2009), 30–37.
    Google ScholarLocate open access versionFindings
  • Su Mon Kywe, Ee-Peng Lim, and Feida Zhu. 2012. A Survey of Recommender Systems in Twitter. In SocInfo. 420–433.
    Google ScholarLocate open access versionFindings
  • Fei-Fei Li, Robert Fergus, and Pietro Perona. 2006. One-Shot Learning of Object Categories. IEEE Trans. Pattern Anal. Mach. Intell. 28, 4 (2006), 594–611.
    Google ScholarLocate open access versionFindings
  • Greg Linden, Brent Smith, and Jeremy York. 2003. Amazon.com Recommendations: Item-to-Item Collaborative Filtering. IEEE Internet Computing 7, 1 (2003), 76–80.
    Google ScholarLocate open access versionFindings
  • Babak Loni, Yue Shi, Martha Larson, and Alan Hanjalic. 2014. Cross-Domain Collaborative Filtering with Factorization Machines. In ECIR. 656–661.
    Google ScholarLocate open access versionFindings
  • Denis A. Lussier and Richard W. Olshavsky. 1979. Task Complexity and Contingent Processing in Brand Choice. Journal of Consumer Research 6, 2 (1979), 154–165.
    Google ScholarLocate open access versionFindings
  • Tong Man, Huawei Shen, Xiaolong Jin, and Xueqi Cheng. 2017. Cross-Domain Recommendation: An Embedding and Mapping Approach. In IJCAI. 2464–2470.
    Google ScholarFindings
  • Tsendsuren Munkhdalai and Hong Yu. 2017. Meta Networks. In ICML. 2554–2563.
    Google ScholarFindings
  • Alex Nichol, Joshua Achiam, and John Schulman. 2018. On First-Order Meta-Learning Algorithms. CoRR abs/1803.02999 (2018).
    Findings
  • Cosimo Palmisano, Alexander Tuzhilin, and Michele Gorgoglione. 2008. Using Context to Improve Predictive Modeling of Customers in Personalization Applications. IEEE Trans. Knowl. Data Eng. 20, 11 (2008), 1535–1549.
    Google ScholarLocate open access versionFindings
  • John W. Payne, James R. Bettman, Eloise Coupey, and Eric J. Johnson. 1992. A constructive process view of decision making: Multiple strategies in judgment and choice. Acta Psychologica 80, 1 (1992), 107 – 141.
    Google ScholarLocate open access versionFindings
  • Bryan Perozzi, Rami Al-Rfou, and Steven Skiena. 2014. DeepWalk: online learning of social representations. In SIGKDD. 701–710.
    Google ScholarLocate open access versionFindings
  • Massimo Quadrana, Paolo Cremonesi, and Dietmar Jannach. 2018. Sequenceaware Recommender Systems. In UMAP. 373–374.
    Google ScholarLocate open access versionFindings
  • Sachin Ravi and Hugo Larochelle. 2017. Optimization as a model for few-shot learning. In ICLR, Vol. 2. 6.
    Google ScholarLocate open access versionFindings
  • Steffen Rendle. 2010. Factorization Machines. In ICDM. 995–1000.
    Google ScholarLocate open access versionFindings
  • Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. 2009. BPR: Bayesian Personalized Ranking from Implicit Feedback. In UAI. 452– 461.
    Google ScholarLocate open access versionFindings
  • Herbert Robbins and Sutton Monro. 1951. A Stochastic Approximation Method. Ann. Math. Statist. 22, 3 (09 1951), 400–407.
    Google ScholarLocate open access versionFindings
  • Ajit Paul Singh and Geoffrey J. Gordon. 2008. Relational learning via collective matrix factorization. In Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Las Vegas, Nevada, USA, August 24-27, 2008. 650–658.
    Google ScholarLocate open access versionFindings
  • Jake Snell, Kevin Swersky, and Richard S. Zemel. 2017. Prototypical Networks for Few-shot Learning. In NIPS. 4080–4090.
    Google ScholarFindings
  • Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: A Simple Way to Prevent Neural Networks from Overfitting. JMLR 15, 1 (Jan. 2014), 1929–1958.
    Google ScholarLocate open access versionFindings
  • Kostas Stefanidis, Evaggelia Pitoura, and Panos Vassiliadis. 2007. A context-aware preference database system. Int. J. Pervasive Computing and Communications 3, 4 (2007), 439–460.
    Google ScholarLocate open access versionFindings
  • Jiaxi Tang and Ke Wang. 2018. Personalized Top-N Sequential Recommendation via Convolutional Sequence Embedding. In WSDM. 565–573.
    Google ScholarLocate open access versionFindings
  • Tijmen Tieleman and Geoffrey Hinton. 2012. Lecture 6.5-rmsprop: Divide the gradient by a running average of its recent magnitude. Technical Report.
    Google ScholarFindings
  • Lisa Torrey and Jude Shavlik. 2009. Transfer Learning.
    Google ScholarFindings
  • Jan N. van Rijn and Frank Hutter. 2018. Hyperparameter Importance Across Datasets. In SIGKDD. 2367–2376.
    Google ScholarFindings
  • Manasi Vartak, Arvind Thiagarajan, Conrado Miranda, Jeshua Bratman, and Hugo Larochelle. 2017. A Meta-Learning Perspective on Cold-Start Recommendations for Items. In NIPS. 6907–6917.
    Google ScholarFindings
  • Oriol Vinyals, Charles Blundell, Tim Lillicrap, Koray Kavukcuoglu, and Daan Wierstra. 2016. Matching Networks for One Shot Learning. In NIPS. 3630–3638.
    Google ScholarFindings
  • Maksims Volkovs, Guang Wei Yu, and Tomi Poutanen. 2017. DropoutNet: Addressing Cold Start in Recommender Systems. In NIPS. 4964–4973.
    Google ScholarFindings
  • Ronald J. Williams. 1992. Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning. Machine Learning 8 (1992), 229–256.
    Google ScholarLocate open access versionFindings
  • Wenhan Xiong, Mo Yu, Shiyu Chang, Xiaoxiao Guo, and William Yang Wang. 2018. One-Shot Relational Learning for Knowledge Graphs. In EMNLP. 1980– 1990.
    Google ScholarFindings
  • Deqing Yang, Jingrui He, Huazheng Qin, Yanghua Xiao, and Wei Wang. 2015. A Graph-based Recommendation across Heterogeneous Domains. In CIKM. 463– 472. 4 https://youtu.be/TNHLZqWnQwc 5 https://pytorch.org/6 http://github.com/hexiangnan/neural_collaborative_filtering 7 https://www.csie.ntu.edu.tw/~cjlin/libmf/8 http://www.libfm.org 9 http://jmcauley.ucsd.edu/data/amazon difference between the source domain and the target domain and fully evaluate the model’s ability to learn user behaviors in new scenarios, we select the leaf categories in different first-order categories as the source domain, meta-training set and meta-testing set. The specific split of first-order categories is displayed in Table 5.
    Locate open access versionFindings
Your rating :
0

 

Tags
Comments