Leveraging Title-Abstract Attentive Semantics for Paper Recommendation

Bowei Chen
Bowei Chen
Xiaoyan Zhang
Xiaoyan Zhang
Zhirong Liu
Zhirong Liu
Zhenhua Dong
Zhenhua Dong

national conference on artificial intelligence, 2020.

Cited by: 1|Bibtex|Views83|Links
Keywords:
collaborative topic regressionmemory networkNormalized Discounted Cumulated Gainuser itemCollaborative filteringMore(16+)
Weibo:
We proposed a two-level attentive neural network called TAAS to capture the semantic correlation between title and abstract for paper recommendation

Abstract:

Paper recommendation is a research topic to provide users with personalized papers of interest. However, most existing approaches equally treat title and abstract as the input to learn the representation of a paper, ignoring their semantic relationship. In this paper, we regard the abstract as a sequence of sentences, and propose a two-le...More

Code:

Data:

0
Introduction
  • Ever-increasing number of research papers have been published over the last decades, resulting in a problem known as ‘information overload’.
  • Researchers have to spend more time searching for articles articles they are interested in.
  • Paper recommendation is more important than before.
  • Collaborative filtering (CF) has been widely adopted in recommendation systems, which explores user-item historical interactions.
  • CF often generates poor performance since the user-item interaction matrix is very sparse in many fields.
  • Auxiliary information is introduced to enhance recommendation performance
Highlights
  • Ever-increasing number of research papers have been published over the last decades, resulting in a problem known as ‘information overload’
  • In this paper we propose a TitleAbstract Attentive Semantic network to capture the semantic relationship between title and abstract for paper recommendation
  • In the word-level sub-network, we propose an attentive Long-Short Term Memory (LSTM) network to learn sentence representation by considering the importance of a word with respect to those in the title
  • To learn fine-grained user preference, we present a keyvalue memory network to memorize the user preference for sequential sentence patterns on the basis of title representation, which overcomes the shortcoming that traditional Gated Recurrent Unit (GRU) networks prone to forgetting effective memory
  • We proposed a two-level attentive neural network called TAAS to capture the semantic correlation between title and abstract for paper recommendation
  • The results show that our TAAS model can consistently and significantly outperform the other models in all ranking metrics
  • Our experimental results on two real datasets have confirmed that the proposed method can reach superior performance to other counterparts
Methods
  • The authors study three research aspects of the model: (1) the effectiveness of the proposed approach in comparison with other state-of-the-art methods; (2) the impact of the two-level attentive sub-networks; and (3) the influence of the semantic weight parameters used in the model.
Results
  • Evaluation Metrics

    The authors adopt four widely taken ranking metrics to evaluate recommendation accuracy of all comparison methods, including Precision at n (Pre@n), Mean Reciprocal Rank (MRR), and Normalized Discounted Cumulated Gain (NDCG).
  • The detailed definitions of these metrics can be found in (Ricci et al 2010).
  • Due to high sparsity of the datasets, the most basic approach BPR, which only takes into account the historical interactions of users and items, produces the worst performance among the four approaches
  • It implies the importance of textual content information in improving performance of paper recommendation
Conclusion
  • The authors proposed a two-level attentive neural network called TAAS to capture the semantic correlation between title and abstract for paper recommendation.
  • The word-level attentive sub-network aimed to generate sentence representations, the assumption behind which is that words appearing in title are more informative than others.
  • The sentence-level attentive sub-network took the title representation as the global memory, which was iteratively updated by abstract sentences, and sequentially memorized by a key-value memory network.
  • The authors' experimental results on two real datasets have confirmed that the proposed method can reach superior performance to other counterparts
Summary
  • Introduction:

    Ever-increasing number of research papers have been published over the last decades, resulting in a problem known as ‘information overload’.
  • Researchers have to spend more time searching for articles articles they are interested in.
  • Paper recommendation is more important than before.
  • Collaborative filtering (CF) has been widely adopted in recommendation systems, which explores user-item historical interactions.
  • CF often generates poor performance since the user-item interaction matrix is very sparse in many fields.
  • Auxiliary information is introduced to enhance recommendation performance
  • Methods:

    The authors study three research aspects of the model: (1) the effectiveness of the proposed approach in comparison with other state-of-the-art methods; (2) the impact of the two-level attentive sub-networks; and (3) the influence of the semantic weight parameters used in the model.
  • Results:

    Evaluation Metrics

    The authors adopt four widely taken ranking metrics to evaluate recommendation accuracy of all comparison methods, including Precision at n (Pre@n), Mean Reciprocal Rank (MRR), and Normalized Discounted Cumulated Gain (NDCG).
  • The detailed definitions of these metrics can be found in (Ricci et al 2010).
  • Due to high sparsity of the datasets, the most basic approach BPR, which only takes into account the historical interactions of users and items, produces the worst performance among the four approaches
  • It implies the importance of textual content information in improving performance of paper recommendation
  • Conclusion:

    The authors proposed a two-level attentive neural network called TAAS to capture the semantic correlation between title and abstract for paper recommendation.
  • The word-level attentive sub-network aimed to generate sentence representations, the assumption behind which is that words appearing in title are more informative than others.
  • The sentence-level attentive sub-network took the title representation as the global memory, which was iteratively updated by abstract sentences, and sequentially memorized by a key-value memory network.
  • The authors' experimental results on two real datasets have confirmed that the proposed method can reach superior performance to other counterparts
Related work
  • Structure-based Paper Recommendation

    The first type of paper recommendation is based on the citation structure, i.e., the papers it cites and those citing it. The constructed paper graph is further mined to calculate paper similarity and generate paper recommendations. For example, (Sugiyama and Kan 2010) construct paper representation based on the TF-IDF technology, which is heavily relied on the term frequency. The similarity between papers based on citation references is used as weights to build user and paper profiles. However, not all relevant works can be fully covered in one paper. To alleviate this issue, (Sugiyama and Kan 2013) further improve their previous model by extending a paper’s reference list with the involvement of the top-N relevant papers. Moreover, (Mohammadi et al 2016) build a basic paper graph based on the reference citations. A random walk algorithm is devised to generate recommendation items. To sum up, the underlying assumption of this research line stresses that the citation topology can accurately reflect paper relatedness. However, in many cases, such an assumption cannot hold because: (1) most recently published papers cannot be referred to by previous papers; (2) some valuable references may be missing due to the unawareness of researchers; and (3) some irrelevant or less relevant papers may be adopted in the reference list, for example, some other papers from the same authors.
Funding
  • This work was supported in part by the National Natural Science Foundation of China under Grant 61972078 and 61702084, and by the Fundamental Research Funds for Central Universities under Grant N181705007
  • It was also partially sponsored by Huawei Innovation Research Program
Reference
  • Burke, R. 2007. Hybrid Web Recommender Systems. Springer Berlin Heidelberg.
    Google ScholarFindings
  • Chen, X.; Xu, H.; Zhang, Y.; Tang, J.; Cao, Y.; Qin, Z.; and Zha, H. 2018. Sequential recommendation with user memory networks. In Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining (WSDM), 108–116.
    Google ScholarLocate open access versionFindings
  • Chen, C.; Zhang, M.; Wang, C.; Ma, W.; Li, M.; Liu, Y.; and Ma, S. 2019. An efficient adaptive transfer neural network for social-aware recommendation. In Proceedings of the 42Nd International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR), 225–234.
    Google ScholarLocate open access versionFindings
  • Guo, G.; Zhang, J.; Sun, Z.; and Yorke-Smith, N. 2015. Librec: A java library for recommender systems. In Latebreaking Results and Workshop Proceedings of the 23rd Conference on User Modelling, Adaptation and Personalization (UMAP).
    Google ScholarLocate open access versionFindings
  • Hassan, H. A. M. 2017. Personalized research paper recommendation using deep learning. In Proceedings of the 25th Conference on User Modeling, Adaptation and Personalization (UMAP), 327–330.
    Google ScholarLocate open access versionFindings
  • He, X.; He, Z.; Du, X.; and Chua, T.-S. 2018. Adversarial personalized ranking for recommendation. In The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval (SIGIR), 355–364.
    Google ScholarLocate open access versionFindings
  • Huang, J.; Zhao, W. X.; Dou, H.; Wen, J.-R.; and Chang, E. Y. 2018. Improving sequential recommendation with knowledge-enhanced memory networks. In The 41st International ACM SIGIR Conference on Research; Development in Information Retrieval (SIGIR), 505–514.
    Google ScholarLocate open access versionFindings
  • Kim, D.; Park, C.; Oh, J.; Lee, S.; and Yu, H. 2016. Convolutional matrix factorization for document context-aware recommendation (recsys). In Proceedings of the 10th ACM Conference on Recommender Systems, 233–240.
    Google ScholarLocate open access versionFindings
  • Mohammadi, S.; Kylasa, S.; Kollias, G.; and Grama, A. 2016. Context-specific recommendation system for predicting similar pubmed articles. In IEEE 16th International Conference on Data Mining Workshops (ICDMW), 1007– 1014.
    Google ScholarLocate open access versionFindings
  • Rendle, S.; Freudenthaler, C.; Gantner, Z.; and SchmidtThieme, L. 2009. Bpr: Bayesian personalized ranking from implicit feedback. In Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence (UAI), 452– 461.
    Google ScholarLocate open access versionFindings
  • Ricci, F.; Rokach, L.; Shapira, B.; and Kantor, P. B. 2010. Recommender Systems Handbook. Berlin, Heidelberg: Springer-Verlag, 1st edition.
    Google ScholarFindings
  • Salakhutdinov, R., and Mnih, A. 2007. Probabilistic matrix factorization. In Proceedings of the 20th International Conference on Neural Information Processing Systems (NIPS), 1257–1264.
    Google ScholarLocate open access versionFindings
  • Sugiyama, K., and Kan, M.-Y. 2010. Scholarly paper recommendation via user’s recent research interests. In Proceedings of the 10th Annual Joint Conference on Digital Libraries (JCDL), 29–38.
    Google ScholarLocate open access versionFindings
  • Sugiyama, K., and Kan, M.-Y. 2013. Exploiting potential citation papers in scholarly paper recommendation. In Proceedings of the 13th ACM/IEEE-CS Joint Conference on Digital Libraries (JCDL), 153–162.
    Google ScholarLocate open access versionFindings
  • Sun, J.; Ma, J.; Liu, X.; Liu, Z.; Wang, G.; Jiang, H.; and Silva, T. 2013. A novel approach for personalized article recommendation in online scientific communities. 46th Hawaii International Conference on System Sciences 1543– 1552.
    Google ScholarFindings
  • Tran, T.; Sweeney, R.; and Lee, K. 2019. Adversarial mahalanobis distance-based attentive song recommender for automatic playlist continuation. In Proceedings of the 42Nd International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR), 245–254.
    Google ScholarLocate open access versionFindings
  • Vincent, P.; Larochelle, H.; Lajoie, I.; Bengio, Y.; and Manzagol, P.-A. 2010. Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion. Journal of Machine Learning Research 3371–3408.
    Google ScholarLocate open access versionFindings
  • Wang, C., and Blei, D. M. 2011. Collaborative topic modeling for recommending scientific articles. In Proceedings of the 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD), 448–456.
    Google ScholarLocate open access versionFindings
  • Wang, H.; Wang, N.; and Yeung, D.-Y. 2015. Collaborative deep learning for recommender systems. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD), 1235–1244.
    Google ScholarLocate open access versionFindings
  • Weston, J.; Chopra, S.; and Bordes, A. 2014. Memory networks. Computing Research Repository.
    Google ScholarFindings
Your rating :
0

 

Tags
Comments