"Alternative Study Designs" For The Evaluation Of Digital Health Applications - A Real Alternative?

Introduction: After the Digital Healthcare Act (Digitale-Versorgung-Gesetz, DVG) reformed digital health applications' (Digitale Gesundheitsanwendungen, DiGAs) access to German Statutory Health Insurance (SHI) reimbursement, the discussion concerning necessary evidence requirements has intensified. In the past, different "alternative study designs" have been proposed to replace randomized controlled trials (RCTs) in the DiGA efficacy and benefit assessments. The present paper examines the suitability of these alternative designs for informing SHI reimbursement decisions.Methods: The four alternative study designs primarily discussed in the context of DiGA - "Continuous Evaluation of Evolving Behavioral Intervention Technologies" (CEEBIT), "Multiphase Optimization Strategy" (MOST), "Sequential Multiple Assignment Randomized Trial" (SMART) and "Micro-Randomized Trial" (MRT) - are characterized and compared on the basis of relevant primary and secondary sources. Subsequently, their suitability for effectiveness and benefit evaluation in the context of SHI reimbursement decisions is discussed.Results: None of the study designs examined aims primarily at conclusively demonstrating efficacy and benefit. Three of the four designs (MOST, SMART, MRT) focus on the development and optimization of interventions. In order to reduce resource requirements, the approaches presented sometimes deviate considerably from the methodological approach in traditional RCTs. This is especially true for their applied statistical error tolerance and their underlying randomization logic. Three of the four concepts (MOST, SMART, MRT) therefore still require RCTs after the development phase in order to demonstrate the effectiveness and benefit of the optimized intervention.Discussion: The methodological differences of the alternative study designs compared to classical RCTs are accompanied by serious potentials for bias and uncertainties with regard to the identified intervention effects. These may be acceptable in the context of intervention development, but do not appear to be appropriate for use in collective SHI reimbursement decisions.Conclusion: The alternative study designs presented cannot be regarded as a suitable RCT alternative for efficacy and benefit assessments. A pragmatic study design, which continues to meet high methodological standards, and better utilization of real-world data could, in the future, contribute to a compromise between the justified claims to sufficient certainty of results on the one hand and appropriate procedural effort on the other.