Graph Neural News Recommendation with Unsupervised Preference Disentanglement

    ACL, pp. 4255-4264, 2020.

    Cited by: 0|Bibtex|Views48|Links
    EI
    Keywords:
    parallel convolutional neural networksbipartite graphnovel graphgraph neural networksgraph neural news recommendationMore(11+)
    Wei bo:
    This work makes the following three contributions: In this work, we model the user-news interactions as a bipartite graph and propose a novel graph neural news recommendation model GNUD with unsupervised preference disentanglement

    Abstract:

    With the explosion of news information, personalized news recommendation has become very important for users to quickly find their interested contents. Most existing methods usually learn the representations of users and news from news contents for recommendation. However, they seldom consider highorder connectivity underlying the user-ne...More

    Code:

    Data:

    0
    Introduction
    • DKN (Wang et al, 2018) learns knowledge-aware news representation via multi-channel CNN and gets a representation of a user by aggregating her clicked news history with different weights
    • These methods (Wu et al, 2019b; Zhu et al, 2019; An et al, 2019) usually focus on news contents, and seldom consider the collaborative signal in the form of high-order connectivity underlying the user-news interactions.
    • Capturing high-order connectivity among users and news could deeply exploit structure characteristics and alleviate the sparsity, improving the rec-
    Highlights
    • The amount of news and articles on many news platforms, such as Google News1, has been growing 1 2 3 ? Layer-2 Layer-1 ⋯

      constantly at an explosive rate, making it difficult for users to seek for news that they are interested in
    • To address the above issues, we model the user-news interactions as a bipartite graph and propose a novel Graph Neural News Recommendation Model with Unsupervised preference Disentanglement (GNUD)
    • This work makes the following three contributions: (1) In this work, we model the user-news interactions as a bipartite graph and propose a novel graph neural news recommendation model GNUD with unsupervised preference disentanglement
    • We consider the high-order connectivity as well as the latent preference factors underlying the user-news interactions, and propose a novel graph neural news recommendation model GNUD with unsupervised preference disentanglement
    • Our model regards the user-news interactions as a bipartite graph and encode high-order relationships among users and news by graph convolution
    • Experimental results on realworld news datasets demonstrate that our model achieves significant performance gains compared to state-of-the-art methods, supporting the importance of exploiting the high-order connectivity and disentangling the latent preference factors in user and news representations
    Methods
    • DKN (Wang et al, 2018), a deep content based news recommendation framework fusing semanticlevel and knowledge-level representations.
    • All the baselines are initialized as the corresponding papers, and in terms of neural network models the authors use the same word embedding dimension for fair comparison.
    • They are carefully tuned to achieve their optimal performance.
    • The authors independently repeat each experiment for 10 times and report the average performance
    Results
    • Experimental results on real-world news datasets demonstrate that the proposed model can effectively improve the performance of news recommendation and outperform state-of-the-art news recommendation methods.
    • (3) Experimental results on real-world datasets demonstrate that the proposed model significantly outperforms state-of-the-art news recommendation methods.
    • The authors can observe that the proposed model GNUD consistently outperforms all the state-of-the-art baseline methods on both datasets.
    • GNUD improves the best deep neural models DKN and DAN more than 6.45% on AUC and 7.79% on F1 on both datasets
    Conclusion
    • The authors consider the high-order connectivity as well as the latent preference factors underlying the user-news interactions, and propose a novel graph neural news recommendation model GNUD with unsupervised preference disentanglement.
    • The learned representations are disentangled with different latent preference factors by a neighborhood routing mechanism, enhancing expressiveness and interpretability.
    • Experimental results on realworld news datasets demonstrate that the model achieves significant performance gains compared to state-of-the-art methods, supporting the importance of exploiting the high-order connectivity and disentangling the latent preference factors in user and news representations
    Summary
    • Introduction:

      DKN (Wang et al, 2018) learns knowledge-aware news representation via multi-channel CNN and gets a representation of a user by aggregating her clicked news history with different weights
    • These methods (Wu et al, 2019b; Zhu et al, 2019; An et al, 2019) usually focus on news contents, and seldom consider the collaborative signal in the form of high-order connectivity underlying the user-news interactions.
    • Capturing high-order connectivity among users and news could deeply exploit structure characteristics and alleviate the sparsity, improving the rec-
    • Methods:

      DKN (Wang et al, 2018), a deep content based news recommendation framework fusing semanticlevel and knowledge-level representations.
    • All the baselines are initialized as the corresponding papers, and in terms of neural network models the authors use the same word embedding dimension for fair comparison.
    • They are carefully tuned to achieve their optimal performance.
    • The authors independently repeat each experiment for 10 times and report the average performance
    • Results:

      Experimental results on real-world news datasets demonstrate that the proposed model can effectively improve the performance of news recommendation and outperform state-of-the-art news recommendation methods.
    • (3) Experimental results on real-world datasets demonstrate that the proposed model significantly outperforms state-of-the-art news recommendation methods.
    • The authors can observe that the proposed model GNUD consistently outperforms all the state-of-the-art baseline methods on both datasets.
    • GNUD improves the best deep neural models DKN and DAN more than 6.45% on AUC and 7.79% on F1 on both datasets
    • Conclusion:

      The authors consider the high-order connectivity as well as the latent preference factors underlying the user-news interactions, and propose a novel graph neural news recommendation model GNUD with unsupervised preference disentanglement.
    • The learned representations are disentangled with different latent preference factors by a neighborhood routing mechanism, enhancing expressiveness and interpretability.
    • Experimental results on realworld news datasets demonstrate that the model achieves significant performance gains compared to state-of-the-art methods, supporting the importance of exploiting the high-order connectivity and disentangling the latent preference factors in user and news representations
    Tables
    • Table1: Statistics of our datasets
    • Table2: The performance of different methods on news recommendation
    • Table3: The performance of GNUD with different layer numbers
    Download tables as Excel
    Related work
    • In this section, we will review the related studies in three aspects, namely news recommendation, graph neural networks and disentangled representation learning.

      News recommendation. Personalized news recommendation is an important task in natural language processing field, which has been widely explored in recent years. Learning better user and news representations is a central task for news recommendation. Traditional collaborative filtering (CF) based methods (Wang and Blei, 2011) often utilize historical interactions between users and news to define the objective function for model training, aiming to predict a personalized ranking over a set of candidates for each user. They usually suffer from cold-start problem since news are often substituted frequently. Many works attempt to take advantage of rich content information, effectively improving the recommendation performance. For example, DSSM (Huang et al, 2013) is a content-based deep neural network to rank a set of documents given a query. Some works (Wang et al, 2018; Zhu et al, 2019) propose to improve news representations via external knowledge, and learn representations of users from their browsed news using an attention module. Wu et al (2019b) applied attention mechanism at both word- and news-level to model different informativeness on news content for different users. Wu et al (2019a) exploited different types of news information with an attentive multi-view learning framework. An et al (2019) considered both titles and topic categories of news, and learned both long- and shortterm user representations, while Wu et al (2019c) represented them by multi-head attention mechanism. However, these works seldom mine highorder structure information.
    Funding
    • This work is supported by the National Natural Science Foundation of China (No U1936220, 61806020, 61772082, 61972047, 61702296), the National Key Research and Development Program of China (2018YFB1402600), the CCF-Tencent Open Fund, and the Fundamental Research Funds for the Central Universities
    Reference
    • Mingxiao An, Fangzhao Wu, Chuhan Wu, Kun Zhang, Zheng Liu, and Xing Xie. 2019. Neural news recommendation with long-and short-term user representations. In ACL, pages 336–345.
      Google ScholarLocate open access versionFindings
    • Yoshua Bengio, Aaron Courville, and Pascal Vincent. 2013. Representation learning: A review and new perspectives. IEEE transactions on pattern analysis and machine intelligence, 35(8):1798–1828.
      Google ScholarLocate open access versionFindings
    • Heng-Tze Cheng, Levent Koc, Jeremiah Harmsen, Tal Shaked, Tushar Chandra, Hrishi Aradhye, Glen Anderson, Greg Corrado, Wei Chai, Mustafa Ispir, et al. 2016.
      Google ScholarLocate open access versionFindings
    • Spyros Gidaris, Praveer Singh, and Nikos Komodakis. 2018. Unsupervised representation learning by predicting image rotations. In ICLR.
      Google ScholarFindings
    • Jon Atle Gulla, Lemei Zhang, Peng Liu, Ozlem Ozgobek, and Xiaomeng Su. 2017. The adressa dataset for news recommendation. In WI, pages 1042–1048.
      Google ScholarLocate open access versionFindings
    • Huifeng Guo, Ruiming Tang, Yunming Ye, Zhenguo Li, and Xiuqiang He. 2017. Deepfm: a factorizationmachine based neural network for ctr prediction. In IJCAI, pages 1725–1731.
      Google ScholarLocate open access versionFindings
    • Will Hamilton, Zhitao Ying, and Jure Leskovec. 201Inductive representation learning on large graphs. In NIPS, pages 1024–1034.
      Google ScholarLocate open access versionFindings
    • Irina Higgins, Loic Matthey, Arka Pal, Christopher Burgess, Xavier Glorot, Matthew Botvinick, Shakir Mohamed, and Alexander Lerchner. 2017. beta-vae: Learning basic visual concepts with a constrained variational framework. In ICLR.
      Google ScholarFindings
    • Jun-Ting Hsieh, Bingbin Liu, De-An Huang, Li F FeiFei, and Juan Carlos Niebles. 2018. Learning to decompose and disentangle representations for video prediction. In NIPS, pages 517–526.
      Google ScholarLocate open access versionFindings
    • Linmei Hu, Chen Li, Chuan Shi, Cheng Yang, and Chao Shao. 2020. Graph neural news recommendation with long-term and short-term interest modeling. Information Processing Management, 57(2):102142.
      Google ScholarLocate open access versionFindings
    • Po-Sen Huang, Xiaodong He, Jianfeng Gao, Li Deng, Alex Acero, and Larry Heck. 2013. Learning deep structured semantic models for web search using clickthrough data. In CIKM, pages 2333–2338.
      Google ScholarLocate open access versionFindings
    • Wouter IJntema, Frank Goossen, Flavius Frasincar, and Frederik Hogenboom. 2010. Ontology-based news recommendation. In EDBT/ICDT Workshops, page 16.
      Google ScholarLocate open access versionFindings
    • Hyunjik Kim and Andriy Mnih. 2018. Disentangling by factorising. In ICML, pages 2654–2663.
      Google ScholarLocate open access versionFindings
    • Yoon Kim. 20Convolutional neural networks for sentence classification. In EMNLP, pages 1746– 1751.
      Google ScholarLocate open access versionFindings
    • Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980.
      Findings
    • Diederik P Kingma and Max Welling. 2013. Autoencoding variational bayes. arXiv preprint arXiv:1312.6114.
      Findings
    • Thomas N Kipf and Max Welling. 2016. Semisupervised classification with graph convolutional networks. In ICLR.
      Google ScholarFindings
    • Jiahui Liu, Peter Dolan, and Elin Rønby Pedersen. 2010. Personalized news recommendation based on click behavior. In IUI, pages 31–40.
      Google ScholarLocate open access versionFindings
    • Jianxin Ma, Peng Cui, Kun Kuang, Xin Wang, and Wenwu Zhu. 2019a. Disentangled graph convolutional networks. In ICML, pages 4212–4221.
      Google ScholarLocate open access versionFindings
    • Jianxin Ma, Chang Zhou, Peng Cui, Hongxia Yang, and Wenwu Zhu. 2019b. Learning disentangled representations for recommendation. In NIPS, pages 5712–5723.
      Google ScholarLocate open access versionFindings
    • Shumpei Okura, Yukihiro Tagami, Shingo Ono, and Akira Tajima. 2017. Embedding-based news recommendation for millions of users. In KDD, pages 1933–1942.
      Google ScholarLocate open access versionFindings
    • Steffen Rendle. 2012. Factorization machines with libfm. ACM Transactions on Intelligent Systems and Technology (TIST), 3(3):57.
      Google ScholarLocate open access versionFindings
    • Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. 2017. Graph attention networks. In ICLR.
      Google ScholarFindings
    • Chong Wang and David M Blei. 2011. Collaborative topic modeling for recommending scientific articles. In KDD, pages 448–456.
      Google ScholarLocate open access versionFindings
    • Hongwei Wang, Fuzheng Zhang, Xing Xie, and Minyi Guo. 2018. Dkn: Deep knowledge-aware network for news recommendation. In WWW, pages 1835– 1844.
      Google ScholarLocate open access versionFindings
    • Xiang Wang, Xiangnan He, Meng Wang, Fuli Feng, and Tat-Seng Chua. 2019. Neural graph collaborative filtering. In SIGIR, pages 165–174.
      Google ScholarLocate open access versionFindings
    • Chuhan Wu, Fangzhao Wu, Mingxiao An, Jianqiang Huang, Yongfeng Huang, and Xing Xie. 2019a. Neural news recommendation with attentive multiview learning. In IJCAI, pages 3863–3869.
      Google ScholarLocate open access versionFindings
    • Chuhan Wu, Fangzhao Wu, Mingxiao An, Jianqiang Huang, Yongfeng Huang, and Xing Xie. 2019b. Npa: Neural news recommendation with personalized attention. In KDD, pages 2576–2584.
      Google ScholarLocate open access versionFindings
    • Chuhan Wu, Fangzhao Wu, Suyu Ge, Tao Qi, Yongfeng Huang, and Xing Xie. 2019c. Neural news recommendation with multi-head selfattention. In EMNLP-IJCNLP, pages 6390–6395.
      Google ScholarLocate open access versionFindings
    • Hong-Jian Xue, Xinyu Dai, Jianbing Zhang, Shujian Huang, and Jiajun Chen. 2017. Deep matrix factorization models for recommender systems. In IJCAI, pages 3203–3209.
      Google ScholarLocate open access versionFindings
    • Cheng Yang, Maosong Sun, Xiaoyuan Yi, and Wenhao Li. 2018. Stylistic chinese poetry generation via unsupervised style disentanglement. In EMNLP, pages 3960–3969.
      Google ScholarLocate open access versionFindings
    • Qiannan Zhu, Xiaofei Zhou, Zeliang Song, Jianlong Tan, and Li Guo. 2019. Dan: Deep attention neural network for news recommendation. In AAAI, volume 33, pages 5973–5980.
      Google ScholarLocate open access versionFindings
    Your rating :
    0

     

    Tags
    Comments