JointMap: Joint Query Intent Understanding For Modeling Intent Hierarchies in E-commerce Search

Ahmadvand Ali
Ahmadvand Ali
Kallumadi Surya
Kallumadi Surya
Javed Faizan
Javed Faizan

SIGIR '20: The 43rd International ACM SIGIR conference on research and development in Information Retrieval Virtual Event China July, 2020, pp. 1509-1512, 2020.

Cited by: 0|Bibtex|Views3|Links
EI
Keywords:
art deep learninglearning modeldeep learning modeljointmap modelinstallation guideMore(14+)
Weibo:
We introduced JointMap, a deep learning model designed for jointly learning two high-level intent tasks on e-commerce search data

Abstract:

An accurate understanding of a user's query intent can help improve the performance of downstream tasks such as query scoping and ranking. In the e-commerce domain, recent work in query understanding focuses on the query to product-category mapping. But, a small yet significant percentage of queries (in our website 1.5% or 33M queries in ...More

Code:

Data:

0
Introduction
  • Query intent understanding is a key step in designing advanced retrieval systems like e-commerce search engines [6].
  • There has been a significant improvement in user intent inference, query understanding remains a major challenge [13].
  • E-commerce search queries have multiple intents associated with them.
  • Ashkan et al [1] categorized search queries for e-commerce websites into commercial and nonc-commercial intents.
  • Zhao et al [14] ignore the non-commercial queries due to small percentage of the search traffic.
  • Commercial queries are queries with purchasing intent, while non-commercial queries cover a wide range of customer services (e.g., “military discounts” and “installation guides”) as shown in Table. 1
Highlights
  • Query intent understanding is a key step in designing advanced retrieval systems like e-commerce search engines [6]
  • Various approaches have been proposed to address query understanding such as 1) considering predefined high-level categories, 2) deploying semisupervised learning with click graphs, 3) considering temporal query intent modeling, 4) understanding word-level user intent, and 5) applying relevance feedback and user behaviors
  • We introduce JointMap, a joint-learning model for high-level user intent prediction
  • We introduced JointMap, a deep learning model designed for jointly learning two high-level intent tasks on e-commerce search data
  • Our results were promising compared to the state-of-the-art deep learning models with an average raise of 2.3% and 10.9% on Macroaveraged F1 in user commercial vs. non-commercial intent and product category mapping, respectively
Methods
  • Dataset Experimental Design.
  • The authors use an SVM model with ngram tf*idf as features to perform distant supervision method due to multiple reasons: 1) SVM is fast and scalable, 2) the features and results are interpretable for supervisors, 3) SVM has proved its effectiveness on text data, 4) SVM provides confidence scores to detect the tricky samples.
  • Two different human annotators were asked to label 540 samples manually.
  • The (Matching, Kappa) scores of (0.98, 0.96) are computed, which is a âĂIJsignificant agreement.âĂİ The category distribution is shown in Figure 2
Results
  • Main Results and Ablation Analysis

    To evaluate the models described in Section 4, 70% of the dataset is used for training, 10% for validation, and 20% for test.
  • Main Results and Ablation Analysis.
  • To evaluate the models described in Section 4, 70% of the dataset is used for training, 10% for validation, and 20% for test.
  • Table 2 summarizes the performance of the models.
  • The results are reported for both commercial vs non-commercial classification and product category mapping.
  • All the improvements are statistically significant using a one-tailed Student’s t-test with a p-value < 0.05
Conclusion
  • The authors introduced JointMap, a deep learning model designed for jointly learning two high-level intent tasks on e-commerce search data.
  • The authors' results were promising compared to the state-of-the-art deep learning models with an average raise of 2.3% and 10.9% on Macroaveraged F1 in user commercial vs non-commercial intent and product category mapping, respectively.
  • The authors' future work includes tuning the JointMap model incorporate contextual information within a session.
  • The presented work advances the state-of-the-art user intent prediction, and lays the groundwork for future research on user intent understanding in e-commerce
Summary
  • Introduction:

    Query intent understanding is a key step in designing advanced retrieval systems like e-commerce search engines [6].
  • There has been a significant improvement in user intent inference, query understanding remains a major challenge [13].
  • E-commerce search queries have multiple intents associated with them.
  • Ashkan et al [1] categorized search queries for e-commerce websites into commercial and nonc-commercial intents.
  • Zhao et al [14] ignore the non-commercial queries due to small percentage of the search traffic.
  • Commercial queries are queries with purchasing intent, while non-commercial queries cover a wide range of customer services (e.g., “military discounts” and “installation guides”) as shown in Table. 1
  • Methods:

    Dataset Experimental Design.
  • The authors use an SVM model with ngram tf*idf as features to perform distant supervision method due to multiple reasons: 1) SVM is fast and scalable, 2) the features and results are interpretable for supervisors, 3) SVM has proved its effectiveness on text data, 4) SVM provides confidence scores to detect the tricky samples.
  • Two different human annotators were asked to label 540 samples manually.
  • The (Matching, Kappa) scores of (0.98, 0.96) are computed, which is a âĂIJsignificant agreement.âĂİ The category distribution is shown in Figure 2
  • Results:

    Main Results and Ablation Analysis

    To evaluate the models described in Section 4, 70% of the dataset is used for training, 10% for validation, and 20% for test.
  • Main Results and Ablation Analysis.
  • To evaluate the models described in Section 4, 70% of the dataset is used for training, 10% for validation, and 20% for test.
  • Table 2 summarizes the performance of the models.
  • The results are reported for both commercial vs non-commercial classification and product category mapping.
  • All the improvements are statistically significant using a one-tailed Student’s t-test with a p-value < 0.05
  • Conclusion:

    The authors introduced JointMap, a deep learning model designed for jointly learning two high-level intent tasks on e-commerce search data.
  • The authors' results were promising compared to the state-of-the-art deep learning models with an average raise of 2.3% and 10.9% on Macroaveraged F1 in user commercial vs non-commercial intent and product category mapping, respectively.
  • The authors' future work includes tuning the JointMap model incorporate contextual information within a session.
  • The presented work advances the state-of-the-art user intent prediction, and lays the groundwork for future research on user intent understanding in e-commerce
Tables
  • Table1: Dataset sample queries and their associated labels
  • Table2: Macro- and Micro- averaged F1 for different models. The improvements reported against LEAM
Download tables as Excel
Reference
  • A. Ashkan, C. L. Clarke, E. Agichtein, and Q. Guo. Classifying and characterizing query intent. In proceedings of ECIR, pages 578–586.
    Google ScholarLocate open access versionFindings
  • P. Bojanowski, E. Grave, A. Joulin, and T. Mikolov. Enriching word vectors with subword information. Transactions of the Association for Computational Linguistics, 5:135–146, 2017.
    Google ScholarLocate open access versionFindings
  • R. Caruana. Multitask learning. Machine learning, 28(1):41–75, 1997.
    Google ScholarLocate open access versionFindings
  • F. Charte and et al. Dealing with difficult minority labels in imbalanced mutilabel data sets. Neurocomputing, 326:39–53, 2019.
    Google ScholarLocate open access versionFindings
  • A. Conneau, H. Schwenk, L. Barrault, and Y. Lecun. Very deep convolutional networks for text classification. arXiv preprint arXiv:1606.01781, 2016.
    Findings
  • W. B. Croft, M. Bendersky, H. Li, and G. Xu. Query representation and understanding workshop. In SIGIR Forum, volume 44, pages 48–53, 2010.
    Google ScholarLocate open access versionFindings
  • J.-W. Ha, H. Pyo, and J. Kim. Large-scale item categorization in e-commerce using multiple recurrent neural networks. In SIGKDD, pages 107–115. ACM, 2016.
    Google ScholarLocate open access versionFindings
  • C. Khatri, R. Goel, B. Hedayatnia, A. Metanillou, A. Venkatesh, R. Gabriel, and A. Mandal. Contextual topic modeling for dialog systems. In 2018 IEEE Spoken Language Technology Workshop (SLT), pages 892–899. IEEE, 2018.
    Google ScholarLocate open access versionFindings
  • T.-Y. Lin, P. Goyal, R. Girshick, K. He, and P. Dollár. Focal loss for dense object detection. In ICCV, pages 2980–2988, 2017.
    Google ScholarLocate open access versionFindings
  • J. Liu, W.-C. Chang, Y. Wu, and Y. Yang. Deep learning for extreme multi-label text classification. In proceedings of SIGIR, pages 115–124, 2017.
    Google ScholarLocate open access versionFindings
  • G. Wang, C. Li, W. Wang, Y. Zhang, D. Shen, X. Zhang, R. Henao, and L. Carin. Joint embedding of words and labels for text classification. arXiv preprint arXiv:1805.04174, 2018.
    Findings
  • J. Wang, J. Tian, and et all. A multi-task learning approach for improving product title compression with user search log data. In proceeding of AAAI, 2018.
    Google ScholarLocate open access versionFindings
  • H. Zhang and et al. Generic intent representation in web search. In proceedings of SIGIR, pages 65–74. ACM, 2019.
    Google ScholarLocate open access versionFindings
  • J. Zhao, H. Chen, and D. Yin. A dynamic product-aware learning model for e-commerce query intent understanding. In CIKM, pages 1843–1852, 2019.
    Google ScholarLocate open access versionFindings
Your rating :
0

 

Tags
Comments