Deep Job Understanding at LinkedIn

Jaewon Yang
Jaewon Yang
Ji Yan
Ji Yan
Shuai Wang
Shuai Wang
Fei Chen
Fei Chen

SIGIR '20: The 43rd International ACM SIGIR conference on research and development in Information Retrieval Virtual Event China July, 2020, pp. 2145-2148, 2020.

Cited by: 0|Bibtex|Views19|Links
EI
Keywords:
job recommendation systemjob postingPointwise Mutual Informationprofessional entityNatural Language ProcessingMore(18+)
Weibo:
We develop job understanding models that take noisy job postings as input and output structured data for easy interpretation

Abstract:

As the world's largest professional network, LinkedIn wants to create economic opportunity for everyone in the global workforce. One of its most critical missions is matching jobs with processionals. Improving job targeting accuracy and hire efficiency align with LinkedIn's Member First Motto. To achieve those goals, we need to understand...More

Code:

Data:

0
Introduction
  • LinkedIn serves as a job marketplace that matches millions of jobs to more than 675 million members.
  • To create economic opportunity for every member of the global workforce, LinkedIn needs to understand the job marketplace precisely.
  • It is challenging to model job postings directly in tasks such as job recommendation and applicant evaluation.
  • To address this challenge, the authors develop job understanding models that take noisy job postings as input and output structured data for easy interpretation.
  • The authors standardize job postings into professional entities that
Highlights
  • LinkedIn serves as a job marketplace that matches millions of jobs to more than 675 million members
  • Job postings usually cover a wide range of topics ranging from company description, job qualifications, benefits to disclaimers
  • We develop job understanding models that take noisy job postings as input and output structured data for easy interpretation
  • We present LinkedIn’s deep job understanding and demonstrate LinkedIn’s job posting flow1 powered by our work
  • We developed a set of job standardization models to identify professional entities from job postings, and used an user feedback loop to continuously improve model performance by retraining it with collected data
Results
  • By applying the deep job understanding models and establishing feedback loops, the authors observed +11 Net Promoter Score (NPS) improvement indicating good user satisfaction, and 30% more job applications because of the better job targeting that aims at putting the job post in front of the right candidates.
Conclusion
  • The authors built the worldâĂŹs largest and most complete professional entity taxonomy to categorize and standardize the professional entities in the job posting.
  • The authors built and presented the customer feedback loop by tracking usersâĂŹ interactive behaviour on current models in LinkedInâĂŹs job posting flow.
  • The authors were able to iteratively adjust the models to let them be more powerful and stay more sensitive to the job market changes.
  • The authors' online A/B test results showed that these user feedback loops improved job posters’ satisfaction, increased the performance of the job understanding models, and led to better matches between jobs and the members
Summary
  • Introduction:

    LinkedIn serves as a job marketplace that matches millions of jobs to more than 675 million members.
  • To create economic opportunity for every member of the global workforce, LinkedIn needs to understand the job marketplace precisely.
  • It is challenging to model job postings directly in tasks such as job recommendation and applicant evaluation.
  • To address this challenge, the authors develop job understanding models that take noisy job postings as input and output structured data for easy interpretation.
  • The authors standardize job postings into professional entities that
  • Results:

    By applying the deep job understanding models and establishing feedback loops, the authors observed +11 Net Promoter Score (NPS) improvement indicating good user satisfaction, and 30% more job applications because of the better job targeting that aims at putting the job post in front of the right candidates.
  • Conclusion:

    The authors built the worldâĂŹs largest and most complete professional entity taxonomy to categorize and standardize the professional entities in the job posting.
  • The authors built and presented the customer feedback loop by tracking usersâĂŹ interactive behaviour on current models in LinkedInâĂŹs job posting flow.
  • The authors were able to iteratively adjust the models to let them be more powerful and stay more sensitive to the job market changes.
  • The authors' online A/B test results showed that these user feedback loops improved job posters’ satisfaction, increased the performance of the job understanding models, and led to better matches between jobs and the members
Related work
  • Named Entity Recognition (NER) is most related to our work given the present knowledge. However, there are a few key distinctions between general NER and our work. First, most neural network (NN) NER models [1, 5, 9] are trained on open domain corpora [11, 14] and focus on recognizing a limited set of entities including person, location, date, and organization. These methods are not designed to recognize professional entities. In stark contrast, few work is focused on recognizing professional entities such as title and skills in job recruiting fields. Goindani, et al [4] recognize industries from job postings by treating it as a classification problem. Qin, et. al. [10] developed a NN model to extract skill entities from the job postings and resumes. Yet, they solely focus on one specific entity extractor, neglecting mutual benefits from other entity extractors. Additionally, there is no feedback loop [3, 17] in their development cycle. This is a major shortcoming because models based on their approaches fail to adapt well with job market fluctuations and changes. To our best knowledge, as an important part of our work, feedback loop has not been applied to the entities standardization problem in the job domain yet.
Reference
  • Alan Akbik, Duncan Blythe, and Roland Vollgraf. 2018. Contextual string embeddings for sequence labeling. In COLING. 1638–1649.
    Google ScholarFindings
  • Tianqi Chen and Carlos Guestrin. 2016. Xgboost: A scalable tree boosting system. In KDD.
    Google ScholarFindings
  • Isabel F Cruz, Cosmin Stroe, and Matteo Palmonari. 2012. Interactive user feedback in ontology matching using signature vectors. In ICDE. IEEE.
    Google ScholarFindings
  • Mahak Goindani, Qiaoling Liu, Josh Chao, and Valentin Jijkoun. 2017. Employer industry classification using job postings. In ICDMW. IEEE, 183–188.
    Google ScholarLocate open access versionFindings
  • Deepak Gupta, Asif Ekbal, and Pushpak Bhattacharyya. 2018. A deep neural network based approach for entity extraction in code-mixed Indian social media text. In LREC.
    Google ScholarFindings
  • Mohit Iyyer, Varun Manjunatha, Jordan Boyd-Graber, and Hal Daumé III. 2015. Deep unordered composition rivals syntactic methods for text classification. In IJCNLP.
    Google ScholarFindings
  • Armand Joulin, Edouard Grave, Piotr Bojanowski, and Tomas Mikolov. 2016. Bag of tricks for efficient text classification. arXiv preprint arXiv:1607.01759 (2016).
    Findings
  • Krishnaram Kenthapadi, Benjamin Le, and Ganesh Venkataraman. 2017. Personalized job recommendation system at linkedin: Practical challenges and lessons learned. In RecSys. 346–347.
    Google ScholarFindings
  • Matthew E Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. arXiv preprint arXiv:1802.05365 (2018).
    Findings
  • Chuan Qin, Hengshu Zhu, Chen Zhu, Tong Xu, Fuzhen Zhuang, Chao Ma, Jingshuai Zhang, and Hui Xiong. 2019. DuerQuiz: A Personalized Question Recommender System for Intelligent Job Interview. In KDD. 2165–2173.
    Google ScholarFindings
  • Erik F Sang and Fien De Meulder. 2003. Introduction to the CoNLL-2003 shared task: Language-independent named entity recognition. arXiv preprint cs/0306050 (2003).
    Google ScholarFindings
  • Baoxu Shi, Shan Li, Jaewon Yang, Mustafa Emre Kazdagli, and Qi He. 2020. Learning to Ask Screening Questions for Job Postings. In SIGIR.
    Google ScholarFindings
  • Baoxu Shi, Jaewon Yang, Tim Weninger, Jing How, and Qi He. 2019. Representation Learning in Heterogeneous Professional Social Networks with Ambiguous Social Connections. In IEEE BigData.
    Google ScholarLocate open access versionFindings
  • Ralph Weischedel, Martha Palmer, Mitchell Marcus, Eduard Hovy, Sameer Pradhan, Lance Ramshaw, Nianwen Xue, Ann Taylor, Jeff Kaufman, Michelle Franchini, et al. 2013. Ontonotes release 5.0 ldc2013t19. LDC 23 (2013).
    Google ScholarLocate open access versionFindings
  • Xunxian Wu, Tong Xu, Hengshu Zhu, Le Zhang, Enhong Chen, and Hui Xiong. 2019. Trend-aware tensor factorization for job skill demand analysis. In IJCAI.
    Google ScholarFindings
  • Tong Xu, Hengshu Zhu, Chen Zhu, Pan Li, and Hui Xiong. 2018. Measuring the popularity of job skills in recruitment market: A multi-criteria approach. In
    Google ScholarLocate open access versionFindings
  • Xiao Yan, Jaewon Yang, Mikhail Obukhov, Lin Zhu, Joey Bai, Shiqi Wu, and Qi He. 2019. Social Skill Validation at LinkedIn. In KDD.
    Google ScholarFindings
Your rating :
0

 

Tags
Comments