What Your Images Reveal: Exploiting Visual Contents for Point-of-Interest Recommendation

    WWW, pp. 391-400, 2017.

    Cited by: 87|Bibtex|Views26|Links
    EI
    Keywords:
    user checkPOI recommendationinterest recommendationpoi recommenderVisual Content Enhanced POI recommendationMore(15+)
    Wei bo:
    We investigate visual contents to advance traditional POI recommender systems

    Abstract:

    The rapid growth of Location-based Social Networks (LBSNs) provides a vast amount of check-in data, which facilitates the study of point-of-interest (POI) recommendation. The majority of the existing POI recommendation methods focus on four aspects, i.e., temporal patterns, geographical influence, social correlations and textual content i...More

    Code:

    Data:

    0
    Introduction
    • As an increasingly popular application of location-based services, location-based social networks (LBSNs), such as Yelp, Instagram and Foursquare, have attracted millions of users.
    • Various POI recommendation methods have been proposed, which mainly study four aspects, i.e., geographical influence, social correlations, temporal patterns and textual content indications [8, 37, 35, 4, 34, 10]
    • These aspects have been proven to be effective for improving POI recommendations
    Highlights
    • As an increasingly popular application of location-based services, location-based social networks (LBSNs), such as Yelp, Instagram and Foursquare, have attracted millions of users
    • In an attempt to solve these two challenges, we propose a novel POI recommendation framework called Visual Content Enhanced POI recommendation (VPOI)
    • We investigate visual contents to advance traditional POI recommender systems
    • To effectively utilizes visual contents, we use Convolutional neural network to extract features from images and use it to guide the learning process of latent user and POI features, which leads to a novel framework Visual Content Enhanced POI recommendation
    • Experimental results show that the proposed framework outperforms representative state-of-the-art POI recommender systems
    • Social dimensions [27], which captures the affliction of users to different groups, may help to capture the common preferences of users in the same group for POI recommendation
    Results
    • It is reported that less than 30% images are explicitly tagged with POIs in Instagram[16], which is consistent with observations from the datasets.
    • The performance for PMF decreases up to 11.25% in terms of precision@10 on CHI 20%
    Conclusion
    • The authors investigate visual contents to advance traditional POI recommender systems.
    • To effectively utilizes visual contents, the authors use CNN to extract features from images and use it to guide the learning process of latent user and POI features, which leads to a novel framework VPOI.
    • The proposed VPOI is a flexible framework that is easy to incorporate geographical influence, social correlations, temporal patterns and textual content indications.
    • As user check-in records are streaming data, another direction is to extend VPOI using streaming recommender system techniques [2, 3]
    Summary
    • Introduction:

      As an increasingly popular application of location-based services, location-based social networks (LBSNs), such as Yelp, Instagram and Foursquare, have attracted millions of users.
    • Various POI recommendation methods have been proposed, which mainly study four aspects, i.e., geographical influence, social correlations, temporal patterns and textual content indications [8, 37, 35, 4, 34, 10]
    • These aspects have been proven to be effective for improving POI recommendations
    • Results:

      It is reported that less than 30% images are explicitly tagged with POIs in Instagram[16], which is consistent with observations from the datasets.
    • The performance for PMF decreases up to 11.25% in terms of precision@10 on CHI 20%
    • Conclusion:

      The authors investigate visual contents to advance traditional POI recommender systems.
    • To effectively utilizes visual contents, the authors use CNN to extract features from images and use it to guide the learning process of latent user and POI features, which leads to a novel framework VPOI.
    • The proposed VPOI is a flexible framework that is easy to incorporate geographical influence, social correlations, temporal patterns and textual content indications.
    • As user check-in records are streaming data, another direction is to extend VPOI using streaming recommender system techniques [2, 3]
    Tables
    • Table1: Performance comparison on NYC and CHI in terms of Precision@5 and Recall@5
    • Table2: Performance comparison on NYC and CHI in terms of Precision@10 and Recall@10
    • Table3: Statistics of the Datasets
    • Table4: Performance comparison on NYC and CHI with 5% cold-start users in terms of Precision@5 and Recall@5. Note that numbers inside parentheses in the table denote the performance reductions compared to the perforamnce without cold-start users in Table 1
    • Table5: Performance comparison on NYC and CHI with 5% cold-start users in terms of Precision@10 and
    Download tables as Excel
    Related work
    • In this section, we will briefly review related works on POI recommendation and visual contents for data mining.

      2.1 POI Recommendation

      POI recommendation, also called location recommendation, has been recognized as an essential task on recommender systems. Existing work on POI recommendation generally focuses on four aspects, i.e., geographical influence, social correlations, temporal patterns and textual content indications [8]. Ye et al [36] introduced POI recommendation on LBSNs and investigated the geographical influence [37] and social influence [35] for POI recommendation. Cheng et al [4] investigated the geographical and social influence through a multi-center Gaussian model. Zhang et al [39] further exploits categorical correlations together with geographical and social correlations. Temporal information has also attracted much attention from researchers. Gao et al [9] investigated the temporal cyclic patterns of check-ins in terms of temporal non-uniformness and temporal consecutiveness. Yuan et al [38] incorporated both temporal cyclic information and geographical information for timeaware POI recommendation. Cheng et al [5] introduced the task of successive personalized POI recommendation in LBSNs with a matrix factorization method. Recently, researchers started to explore the textual content information on LBSNs for POI recommendation. Yang et al [34] introduced sentiment information and reported its better performance over state-of-the-art approaches. Liu et al [18] studied the effect of POI-associated tags with an aggregated LDA model. Gao et al [10] studied document content information on LBSNs w.r.t. POI properties, user interests, and sentiment indications. Though various aspects are investigated for POI recommendation, image contents haven’t been studied for POI recommendation while image contents, i.e., visual contents, have been proven to be effective for many data mining tasks, which will be introduced next.
    Funding
    • This material is based upon work supported by, or in part by, the NSF grants #1614576 and IIS-1217466, and the ONR grant N00014-16-1-2257
    Reference
    • Jake Bouvrie. Notes on convolutional neural networks. 2006.
      Google ScholarFindings
    • Shiyu Chang, Yang Zhang, Jiliang Tang, Dawei Yin, Yi Chang, Mark A Hasegawa-Johnson, and Thomas S Huang. Positive-unlabeled learning in streaming networks. In SIGKDD, pages 755–764. ACM, 2016.
      Google ScholarLocate open access versionFindings
    • Shiyu Chang, Yang Zhang, Jiliang Tang, Dawei Yin, Yi Chang, Mark A Hasegawa-Johnson, and Thomas S Huang. Streaming recommender systems. arXiv preprint arXiv:1607.06182, 2016.
      Findings
    • Chen Cheng, Haiqin Yang, Irwin King, and Michael R Lyu. Fused matrix factorization with geographical and social influence in location-based social networks. In Proceedings of the AAAI, 2012.
      Google ScholarLocate open access versionFindings
    • Chen Cheng, Haiqin Yang, Michael R Lyu, and Irwin King. Where you like to go next: Successive point-of-interest recommendation. In IJCAI, 2013.
      Google ScholarFindings
    • David J Crandall, Lars Backstrom, Daniel Huttenlocher, and Jon Kleinberg. Mapping the world’s photos. In WWW, pages 761–770. ACM, 2009.
      Google ScholarLocate open access versionFindings
    • Navneet Dalal and Bill Triggs. Histograms of oriented gradients for human detection. In CVPR, volume 1, pages 886–893. IEEE, 2005.
      Google ScholarLocate open access versionFindings
    • Huiji Gao and Huan Liu. Mining human mobility in location-based social networks. Synthesis Lectures on Data Mining and Knowledge Discovery, 7(2), 2015.
      Google ScholarLocate open access versionFindings
    • Huiji Gao, Jiliang Tang, Xia Hu, and Huan Liu. Exploring temporal effects for location recommendation on location-based social networks. In Proceedings of the Recommender Systems, pages 93–100. ACM, 2013.
      Google ScholarLocate open access versionFindings
    • Huiji Gao, Jiliang Tang, Xia Hu, and Huan Liu. Content-aware point of interest recommendation on location-based social networks. In AAAI, pages 1721–1727.
      Google ScholarLocate open access versionFindings
    • James Hays and Alexei A Efros. Im2gps: estimating geographic information from a single image. In Proceedings of the CVPR, pages 1–8. IEEE, 2008.
      Google ScholarLocate open access versionFindings
    • Kaiming He and Jian Sun. Convolutional neural networks at constrained time cost. In CVPR, pages 5353–5360, 2015.
      Google ScholarLocate open access versionFindings
    • Ruining He and Julian McAuley. Ups and downs: Modeling the visual evolution of fashion trends with one-class collaborative filtering. In Proceedings of the 25th International Conference on World Wide Web, pages 507–517. International World Wide Web Conferences Steering Committee, 2016.
      Google ScholarLocate open access versionFindings
    • Ruining He and Julian McAuley. Vbpr: Visual bayesian personalized ranking from implicit feedback. In Proceedings of the AAAI, 2016.
      Google ScholarLocate open access versionFindings
    • Daniel D Lee and H Sebastian Seung. Algorithms for non-negative matrix factorization. In Advances in neural information processing systems, pages 556–562, 2001.
      Google ScholarLocate open access versionFindings
    • Xutao Li, Tuan-Anh Nguyen Pham, Gao Cong, Quan Yuan, Xiao-Li Li, and Shonali Krishnaswamy. Where you instagram?: Associating your instagram photos with points of interest. In CIKM, 2015.
      Google ScholarLocate open access versionFindings
    • Bin Liu, Yanjie Fu, Zijun Yao, and Hui Xiong. Learning geographical preferences for point-of-interest recommendation. In Proceedings of SIGKDD, pages 1043–1051. ACM, 2013.
      Google ScholarLocate open access versionFindings
    • Bin Liu and Hui Xiong. Point-of-interest recommendation in location based social networks with topic and location awareness. In SDM. SIAM, 2013.
      Google ScholarLocate open access versionFindings
    • Bo Liu, Quan Yuan, Gao Cong, and Dong Xu. Where your photo is taken: Geolocation prediction for social images. Journal of the Association for Information Science and Technology, 65(6):1232–1243, 2014.
      Google ScholarLocate open access versionFindings
    • David G Lowe. Distinctive image features from scale-invariant keypoints. International journal of computer vision, 60(2):91–110, 2004.
      Google ScholarLocate open access versionFindings
    • Julian McAuley, Christopher Targett, Qinfeng Shi, and Anton van den Hengel. Image-based recommendations on styles and substitutes. In Proceedings of the SIGIR, pages 43–52. ACM, 2015.
      Google ScholarLocate open access versionFindings
    • Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. Distributed representations of words and phrases and their compositionality. In Proceedings of the NIPS, pages 3111–3119, 2013.
      Google ScholarLocate open access versionFindings
    • Rong Pan and Martin Scholz. Mind the gaps: weighting the unknown in large-scale one-class collaborative filtering. In Proceedings of SIGKDD, pages 667–676. ACM, 2009.
      Google ScholarLocate open access versionFindings
    • Ali Razavian, Hossein Azizpour, Josephine Sullivan, and Stefan Carlsson. Cnn features off-the-shelf: an astounding baseline for recognition. In Proceedings of the CVPR Workshops, pages 806–813, 2014.
      Google ScholarLocate open access versionFindings
    • Ruslan Salakhutdinov and Andriy Mnih. Probabilistic matrix factorization. In NIPS, volume 20, pages 1–8, 2011.
      Google ScholarLocate open access versionFindings
    • Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014.
      Findings
    • Jiliang Tang, Suhang Wang, Xia Hu, Dawei Yin, Yingzhou Bi, Yi Chang, and Huan Liu. Recommendation with social dimensions. In AAAI, pages 251–257, 2016.
      Google ScholarLocate open access versionFindings
    • Suhang Wang, Jiliang Tang, Charu Aggarwal, and Huan Liu. Linked document embedding for classification. In CIKM, pages 115–124. ACM, 2016.
      Google ScholarLocate open access versionFindings
    • Suhang Wang, Jiliang Tang, and Huan Liu. Toward dual roles of users in recommender systems. In CIKM, pages 1651–1660, 2015.
      Google ScholarLocate open access versionFindings
    • Suhang Wang, Jiliang Tang, Yilin Wang, and Huan Liu. Exploring implicit hierarchical structures for recommender systems. In IJCAI, pages 1813–1819, 2015.
      Google ScholarLocate open access versionFindings
    • Yilin Wang, Suhang Wang, Jiliang Tang, Huan Liu, and Baoxin Li. Unsupervised sentiment analysis for social media images. In IJCAI, pages 2378–2379, 2015.
      Google ScholarLocate open access versionFindings
    • Yilin Wang, Suhang Wang, Jiliang Tang, Huan Liu, and Baoxin Li. Ppp: Joint pointwise and pairwise image label prediction. In CVPR, pages 6005–6013, 2016.
      Google ScholarLocate open access versionFindings
    • Yilin Wang, Suhang Wang, Jiliang Tang, Guojun Qi, Huan Liu, and Baoxin Li. Clare: A joint approach to label classification and tag recommendation. In AAAI, 2017.
      Google ScholarLocate open access versionFindings
    • Dingqi Yang, Daqing Zhang, Zhiyong Yu, and Zhu Wang. A sentiment-enhanced personalized location recommendation system. In Proceedings of the 24th ACM Conference on Hypertext and Social Media, pages 119–128. ACM, 2013.
      Google ScholarLocate open access versionFindings
    • M Ye, X Liu, and W Lee. Exploring social influence for recommendation-a probabilistic generative approach. In Proceedings of SIGIR, pages 325–334, 2012.
      Google ScholarLocate open access versionFindings
    • Mao Ye, Peifeng Yin, and Wang-Chien Lee. Location recommendation for location-based social networks. In Proceedings of the 18th SIGSPATIAL international conference on advances in geographic information systems, pages 458–461. ACM, 2010.
      Google ScholarLocate open access versionFindings
    • Mao Ye, Peifeng Yin, Wang-Chien Lee, and Dik-Lun Lee. Exploiting geographical influence for collaborative point-of-interest recommendation. In Proceedings of SIGIR, pages 325–334. ACM, 2011.
      Google ScholarLocate open access versionFindings
    • Quan Yuan, Gao Cong, Zongyang Ma, Aixin Sun, and Nadia Magnenat Thalmann. Time-aware point-of-interest recommendation. In Proceedings of the 36th international ACM SIGIR conference on Research and development in information retrieval, pages 363–372. ACM, 2013.
      Google ScholarLocate open access versionFindings
    • Jia-Dong Zhang and Chi-Yin Chow. Geosoca: Exploiting geographical, social and categorical correlations for point-of-interest recommendations. In Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 443–452. ACM, 2015.
      Google ScholarLocate open access versionFindings
    • Dequan Zhou, Bin Wang, Seyyed Mohammadreza Rahimi, and Xin Wang. A study of recommending locations on location-based social network by collaborative filtering. In Advances in Artificial Intelligence, pages 255–266.
      Google ScholarLocate open access versionFindings
    Your rating :
    0

     

    Tags
    Comments