Diamonds in the Rough: Leveraging Click Data to Spotlight Underrated Products

Problem Definition. Inspired by a real data set from the Chinese retailer JD.com, we study the click and purchase behavior of customers in an online retail setting by employing a structural estimation approach. In particular, we aim to understand what characteristics of products drive the click versus purchase decisions of customers. With the boom in e-commerce, which has been further fueled due to the current COVID-19 pandemic, an ever increasing number of customers are shopping online. Such a rapid growth in online shopping provides retailers with abundance of data to study customer shopping behavior. For example, online retailers can benefit from a better understanding of the customer search (click) and purchase behavior to improve their operational decisions such as assortment planning. From an academic standpoint, even though there is an extensive body of work on consumer search models, most prior work do not incorporate search sequences and purchase decisions of customers jointly in their analysis. We, however, combine the click and order data to estimate both the click and purchase behavior of customers. Methodology/Results. Using a large data set from JD.com, we propose a structural model to estimate the click and purchase behavior of customers according to a dynamic discrete choice model. In particular, we assume that the customer's utility from each product has an observed and (pre-click) unobserved part (in addition to a random shock). The observed part of the utility is known to the customer prior to the click; however, the unobserved part of the utility can only be learned after the customer clicks on the product. We assume that the customer solves a dynamic program in order to find the optimal search strategy. Due to the curse of dimensionality, we propose a novel value function approximation scheme inspired by the Conditional Choice Probability approach. This, in turn, reduces the estimation into a computationally tractable two-stage process. By combining the click and order data, our proposed structural framework allows us to disentangle and estimate the observed and unobserved parts of product utilities. Our estimation results show that the value of click for customers can be quite significant. This is evidenced by the fact that the unobserved utilities of products vary significantly across products. Most importantly, we are able to identify underrated products which we call diamonds in the rough: these are products with low “face” values (i.e., low observed utilities), but high total utilities due to their high unobserved utilities. Thus, even though such products have a low chance of being clicked (due to their low observed utilities), they have a high chance of being purchased, if clicked. Managerial Implications. Our structural framework provides an online retailer with new tools and insights to better manage the product assortment based on customer click and purchase behavior. In particular, our structural model allows the retailer to disentangle the observed and unobserved parts of product utilities and identify underrated diamond-in-the-rough products. The online retailer can increase her revenue by bringing such products into the spotlight by promoting them on the search page or using tags such as “spotlight product” (similar to “Amazon's Choice” tags on Amazon.com) to entice customers to click on them.