AI helps you reading Science

AI generates interpretation videos

AI extracts and analyses the key points of the paper to generate videos automatically


pub
Go Generating

AI Traceability

AI parses the academic lineage of this thesis


Master Reading Tree
Generate MRT

AI Insight

AI extracts a summary of this paper


Weibo:
We proposed PinSage, a random-walk graph convolutional network

Graph Convolutional Neural Networks for Web-Scale Recommender Systems.

KDD, (2018): 974-983

Cited by: 1145|Views1344
EI

Abstract

Recent advancements in deep neural networks for graph-structured data have led to state-of-the-art performance on recommender system benchmarks. However, making these methods practical and scalable to web-scale recommendation tasks with billions of items and hundreds of millions of users remains an unsolved challenge. Here we describe a l...More

Code:

Data:

0
Introduction
  • Deep learning methods have an increasingly critical role in recommender system applications, being used to learn useful lowdimensional embeddings of images, text, and even individual users [9, 12].
  • Recent years have seen significant developments in this space— especially the development of new deep learning methods that are capable of learning on graph-structured data, which is fundamental for recommendation applications [6, 19, 21, 24, 29, 30].
  • These gains on benchmark tasks have yet to be translated to gains in real-world production environments
Highlights
  • Deep learning methods have an increasingly critical role in recommender system applications, being used to learn useful lowdimensional embeddings of images, text, and even individual users [9, 12]
  • Most prominent among these recent advancements is the success of deep learning architectures known as Graph Convolutional Networks (GCNs) [19, 21, 24, 29]
  • Our work demonstrates the substantial impact that Graph Convolutional Network have on recommendation performance in a real-world environment
  • We evaluate the methods using Mean Reciprocal Rank (MRR), which takes into account of the rank of the item j among recommended items for query item q: Mean Reciprocal Rank = n (q,i)∈ L
  • We proposed PinSage, a random-walk graph convolutional network (GCN)
  • Our work demonstrates the impact that graph convolutional methods can have in a production recommender system, and we believe that PinSage can be further extended in the future to tackle other graph representation learning problems at large scale, including knowledge graph reasoning and graph clustering
Methods
  • The authors describe the technical details of the PinSage architecture and training, as well as a MapReduce pipeline to efficiently generate embeddings using a trained PinSage model.

    The key computational workhorse of the approach is the notion of localized graph convolutions.1 To generate the embedding for a node, the authors apply multiple convolutional modules that aggregate feature information from the node’s local graph neighborhood (Figure 1).
  • Each module learns how to aggregate information from a small graph neighborhood, and by stacking multiple such modules, the approach can gain information about the local network topology.
  • The authors define the hit-rate as the fraction of queries q where i was ranked among the top K of the test sample
  • This metric directly measures the probability that recommendations made by the algorithm contain the items related to the query pin q.
Results
  • Through extensive offline metrics, controlled user studies, and A/B tests, the authors show that the approach achieves state-of-the-art performance compared to other scalable deep content-based recommendation algorithms, in both an item-item recommendation task, as well as a “homefeed” recommendation task.
  • In offline ranking metrics the authors improve over the best performing baseline by more than 40%, in head-to-head human evaluations the recommendations are preferred about 60% of the time, and the A/B tests show 30% to 100% improvements in user engagement across various settings.
  • The authors introduce a number of new training techniques to improve performance and a MapReduce inference pipeline to scale up to graphs with billions of nodes
Conclusion
  • PinSage is a highly-scalable GCN algorithm capable of learning embeddings for nodes in web-scale graphs containing billions of objects.
  • The authors deployed PinSage at Pinterest and comprehensively evaluated the quality of the learned embeddings on a number of recommendation tasks, with offline metrics, user studies and A/B tests all demonstrating a substantial improvement in recommendation performance.
  • The authors' work demonstrates the impact that graph convolutional methods can have in a production recommender system, and the authors believe that PinSage can be further extended in the future to tackle other graph representation learning problems at large scale, including knowledge graph reasoning and graph clustering
Tables
  • Table1: Hit-rate and MRR for PinSage and content-based deep learning baselines. Overall, PinSage gives 150% improvement in hit rate and 60% improvement in MRR over the best baseline.5
  • Table2: Head-to-head comparison of which image is more relevant to the recommended query image
  • Table3: Runtime comparisons for different batch sizes
  • Table4: Performance tradeoffs for importance pooling
Download tables as Excel
Related work
  • Our work builds upon a number of recent advancements in deep learning methods for graph-structured data.

    The notion of neural networks for graph data was first outlined in Gori et al (2005) [15] and further elaborated on in Scarselli et al (2009) [27]. However, these initial approaches to deep learning on graphs required running expensive neural “message-passing” algorithms to convergence and were prohibitively expensive on large graphs. Some limitations were addressed by Gated Graph Sequence Neural Networks [22]—which employs modern recurrent neural architectures—but the approach remains computationally expensive and has mainly been used on graphs with <10, 000 nodes.

    More recently, there has been a surge of methods that rely on the notion of “graph convolutions” or Graph Convolutional Networks (GCNs). This approach originated with the work of Bruna et al (2013), which developed a version of graph convolutions based on spectral graph thery [7]. Following on this work, a number of authors proposed improvements, extensions, and approximations of these spectral convolutions [6, 10, 11, 13, 18, 21, 24, 29, 31], leading to new state-of-the-art results on benchmarks such as node classification, link prediction, as well as recommender system tasks (e.g., the MovieLens benchmark [24]). These approaches have consistently outperformed techniques based upon matrix factorization or random walks (e.g., node2vec [17] and DeepWalk [26]), and their success has led to a surge of interest in applying GCN-based methods to applications ranging from recommender systems [24] to drug design [20, 31]. Hamilton et al (2017b) [19] and Bronstein et al (2017) [6] provide comprehensive surveys of recent advancements.
Funding
  • Through extensive offline metrics, controlled user studies, and A/B tests, we show that our approach achieves state-of-the-art performance compared to other scalable deep content-based recommendation algorithms, in both an item-item recommendation task (i.e., related-pin recommendation), as well as a “homefeed” recommendation task
  • In offline ranking metrics we improve over the best performing baseline by more than 40%, in head-to-head human evaluations our recommendations are preferred about 60% of the time, and the A/B tests show 30% to 100% improvements in user engagement across various settings
  • We also introduce a number of new training techniques to improve performance and a MapReduce inference pipeline to scale up to graphs with billions of nodes
Study subjects and analysis
pairs: 1200000000
We use all other pins as negative items (and sample them as described in Section 3.3). Overall, we use 1.2 billion pairs of positive training examples (in addition to 500 negative examples per batch and 6 hard negative examples per pin). Thus in total we use 7.5 billion training examples

Reference
  • M. Abadi, A. Agarwal, P. Barham, E. Brevdo, Z. Chen, C. Citro, G. S. Corrado, A. Davis, J. Dean, M. Devin, et al. 2016. Tensorflow: Large-scale machine learning on heterogeneous distributed systems. arXiv preprint arXiv:1603.04467 (2016).
    Findings
  • A. Andoni and P. Indyk. 2006. Near-optimal hashing algorithms for approximate nearest neighbor in high dimensions. In FOCS.
    Google ScholarFindings
  • T. Bansal, D. Belanger, and A. McCallum. 2016. Ask the GRU: Multi-task learning for deep text recommendations. In RecSys. ACM.
    Google ScholarFindings
  • Y. Bengio, J. Louradour, R. Collobert, and J. Weston. 2009. Curriculum learning. In ICML.
    Google ScholarFindings
  • A. Z. Broder, D. Carmel, M. Herscovici, A. Soffer, and J. Zien. 2003. Efficient query evaluation using a two-level retrieval process. In CIKM.
    Google ScholarFindings
  • M. M. Bronstein, J. Bruna, Y. LeCun, A. Szlam, and P. Vandergheynst. 2017. Geometric deep learning: Going beyond euclidean data. IEEE Signal Processing Magazine 34, 4 (2017).
    Google ScholarLocate open access versionFindings
  • J. Bruna, W. Zaremba, A. Szlam, and Y. LeCun. 2014. Spectral networks and locally connected networks on graphs. In ICLR.
    Google ScholarFindings
  • J. Chen, T. Ma, and C. Xiao. 201FastGCN: Fast Learning with Graph Convolutional Networks via Importance Sampling. ICLR (2018).
    Google ScholarLocate open access versionFindings
  • P. Covington, J. Adams, and E. Sargin. 2016. Deep neural networks for youtube recommendations. In RecSys. ACM.
    Google ScholarFindings
  • H. Dai, B. Dai, and L. Song. 2016. Discriminative Embeddings of Latent Variable Models for Structured Data. In ICML.
    Google ScholarFindings
  • M. Defferrard, X. Bresson, and P. Vandergheynst. 2016. Convolutional neural networks on graphs with fast localized spectral filtering. In NIPS.
    Google ScholarFindings
  • A. Van den Oord, S. Dieleman, and B. Schrauwen. 2013. Deep content-based music recommendation. In NIPS.
    Google ScholarFindings
  • D. Duvenaud, D. Maclaurin, J. Iparraguirre, R. Bombarell, T. Hirzel, A. Aspuru-Guzik, and R. P. Adams. 2015. Convolutional networks on graphs for learning molecular fingerprints. In NIPS.
    Google ScholarFindings
  • C. Eksombatchai, P. Jindal, J. Z. Liu, Y. Liu, R. Sharma, C. Sugnet, M. Ulrich, and J. Leskovec. 2018. Pixie: A System for Recommending 3+ Billion Items to 200+ Million Users in Real-Time. WWW (2018).
    Google ScholarLocate open access versionFindings
  • M. Gori, G. Monfardini, and F. Scarselli. 2005. A new model for learning in graph domains. In IEEE International Joint Conference on Neural Networks.
    Google ScholarLocate open access versionFindings
  • P. Goyal, P. Dollár, R. Girshick, P. Noordhuis, L. Wesolowski, A. Kyrola, A. Tulloch, Y. Jia, and K. He. 2017. Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour. arXiv preprint arXiv:1706.02677 (2017).
    Findings
  • A. Grover and J. Leskovec. 2016. node2vec: Scalable feature learning for networks. In KDD.
    Google ScholarFindings
  • W. L. Hamilton, R. Ying, and J. Leskovec. 2017. Inductive Representation Learning on Large Graphs. In NIPS.
    Google ScholarFindings
  • W. L. Hamilton, R. Ying, and J. Leskovec. 2017. Representation Learning on Graphs: Methods and Applications. IEEE Data Engineering Bulletin (2017).
    Google ScholarLocate open access versionFindings
  • S. Kearnes, K. McCloskey, M. Berndl, V. Pande, and P. Riley. 2016. Molecular graph convolutions: moving beyond fingerprints. CAMD 30, 8.
    Google ScholarLocate open access versionFindings
  • T. N. Kipf and M. Welling. 2017. Semi-supervised classification with graph convolutional networks. In ICLR.
    Google ScholarFindings
  • Y. Li, D. Tarlow, M. Brockschmidt, and R. Zemel. 2015. Gated graph sequence neural networks. In ICLR.
    Google ScholarFindings
  • T. Mikolov, I Sutskever, K. Chen, G. S. Corrado, and J. Dean. 2013. Distributed representations of words and phrases and their compositionality. In NIPS.
    Google ScholarFindings
  • F. Monti, M. M. Bronstein, and X. Bresson. 2017. Geometric matrix completion with recurrent multi-graph neural networks. In NIPS.
    Google ScholarFindings
  • OpenMP Architecture Review Board. 2015. OpenMP Application Program Interface Version 4.5. (2015).
    Google ScholarFindings
  • B. Perozzi, R. Al-Rfou, and S. Skiena. 2014. DeepWalk: Online learning of social representations. In KDD.
    Google ScholarFindings
  • F. Scarselli, M. Gori, A.C. Tsoi, M. Hagenbuchner, and G. Monfardini. 2009. The graph neural network model. IEEE Transactions on Neural Networks 20, 1 (2009), 61–80.
    Google ScholarLocate open access versionFindings
  • K. Simonyan and A. Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014).
    Findings
  • R. van den Berg, T. N. Kipf, and M. Welling. 2017. Graph Convolutional Matrix Completion. arXiv preprint arXiv:1706.02263 (2017).
    Findings
  • J. You, R. Ying, X. Ren, W. L. Hamilton, and J. Leskovec. 2018. GraphRNN: Generating Realistic Graphs using Deep Auto-regressive Models. ICML (2018).
    Google ScholarLocate open access versionFindings
  • M. Zitnik, M. Agrawal, and J. Leskovec. 2018. Modeling polypharmacy side effects with graph convolutional networks. Bioinformatics (2018).
    Google ScholarLocate open access versionFindings
Your rating :
0

 

Tags
Comments
数据免责声明
页面数据均来自互联网公开来源、合作出版商和通过AI技术自动分析结果,我们不对页面数据的有效性、准确性、正确性、可靠性、完整性和及时性做出任何承诺和保证。若有疑问,可以通过电子邮件方式联系我们:report@aminer.cn
小科