Design and Evaluation of Optimal Free Trials

Free trial promotions are a commonly used customer acquisition strategy in the Software as a Service industry. We use data from a large-scale field experiment to study the effect of trial length on customer-level outcomes. We find that, on average, shorter trial lengths (surprisingly) maximize customer acquisition, retention, and profitability. Next, we examine the mechanism through which trial length affects conversions and rule out the demand cannibalization theory, find support for the consumer learning hypothesis, and show that long stretches of inactivity at the end of the trial are associated with lower conversions. We then develop a personalized targeting policy that allocates the optimal treatment to each user based on individual-level predictions of the outcome of interest (e.g., subscriptions) using a lasso model. We evaluate this policy using the inverse propensity score reward estimator and show that it leads to 6.8% improvement in subscription compared with a uniform 30-days for-all policy. It also performs well on long-term customer retention and revenues in our setting. Further analysis of this policy suggests that skilled and experienced users are more likely to benefit from longer trials, whereas beginners are more responsive to shorter trials. Finally, we show that personalized policies do not always outperform uniform policies, and we should be careful when designing and evaluating personalized policies. In our setting, personalized policies based on other methods (e.g., causal forests, random forests) perform worse than a simple uniform policy that assigns a short trial length to all users.