Déjà Vu: The Importance of Time and Causality in Recommender Systems

Time plays a key role in recommendation. Handling it properly is especially critical when using recommender systems in real-world applications, which may not be as clear when doing research with historical data. In this talk, we will discuss some of the important challenges of handling time in recommendation algorithms at Netflix. We will focus on challenges related to how our users, items, and systems all change over time. We will then discuss some strategies for tackling these challenges, which revolves around proper treatment of causality in our systems. One clear way that time impacts recommendations is that users and items change over time. Users and items come and go from a service. Individual user interests also change, which could be someone picking up a new interest or even just people maturing and their tastes changing. Many recommendation approaches are based on historical co-occurrence, in which case they tend to lag with such changes instead of anticipating them. An item that is new to the system may start cold but then develop a lot of interest because it is new. So the level of interest in an item fluctuates across time, which can be additionally impacted by trends such as external events and seasonality. Beyond these, the system itself changes over time. Feedback loops in the system, where users actions are influenced by the output of the recommendation system, cause the data that the system uses for recommendations to evolve over time. These feedback loops also can degrade the quality of the recommendations because it becomes hard to tease apart when a user is playing something because they enjoy it versus when they are playing it because it was shown prominently. These loops also can make it difficult for new algorithms to be trained and evaluated due to the influence of the production system. Different components of the system also can change over time, which can impact the data seen by other components. Time also becomes a tradeoff for how accurate a model can be against the time it takes to learn or compute recommendations for a certain request. A model giving very promising results offline but taking weeks to learn might be outperformed online by a model with weaker results but more responsive. On the other hand, changing fast may improve online accuracy by being responsive but may also confuse users when everything changes quickly. Since time is fundamental to the recommendation problem, we seek to optimize our handling of time in our systems. This includes designing experimentation around time by splitting across time and avoiding time-traveling data. It means introducing controlled stochasticity and counterfactuals to break feedback loops and understand causality. It means treating time as a first-class citizen in algorithms. It also means that we often want to optimize our recommendation systems such that a user minimizes the time a user needs to interact with it so that they can find something great to watch.