Leveraging Causal Inference to Measure the Impact of a Mental Health App on Users'Well-being

As stated in the United Nations' Sustainable Development Goals, poor mental well-being is one of the biggest problems we are facing worldwide. One possible way of addressing it is through interventions delivered via digital devices since they are scalable, ubiquitous and inexpensive. This is also confirmed by the ever-growing plethora of e-health mobile apps being developed. Although these apps rely to some extent on scientific bases, there is still much work to do to understand the effect of specific digital interventions on app users. To shed light on these effects, we ask what types of interventions within the app have the most significant impact on well-being, and to what extent longer engagement leads to improved outcomes. These questions could be answered with dedicated Randomized Controlled Trials (RCTs), which are generally expensive, time-consuming, and single-purposed. To overcome these difficulties, we adopt instrumental variables on a combination of data collected in an RCT, behavioural data from the app, and a randomized recommender system, to evaluate intervention and app dose-response effects on users' self-reported well-being. Thus, we present a general causal inference approach for extending results from collected data in RCTs applied in the context of digital health intervention. Following this approach, we show how to measure the impact of different types of activities on the users' well-being. This allows us to identify the most impactful activities in the app (namely, sleep and relaxation activities), which have direct implications for the app design. On the other hand, we prove the positive effect of longer app usage.