A Blueprint for Causal Inference in Implementation Systems

Background: Following a decade of significant progress in implementation science, research efforts are increasingly focused on the investigation of implementation mechanisms and the adoption of multiple EBPs in service delivery systems. This calls for the development of a systems approach to learn about causal mechanisms in implementation systems and a formal methodology for evaluation of systems of care. Methods: We develop a multilevel decision juncture approach to study implementation systems based on advances in implementation research and methods developed in the literature on causal inference and program evaluation. Implementation phases are linked through a sequence of decisions made by relevant agents in the systems at various levels. Formulating the decision juncture as a system of structural equations, each describing a causal relationship between variables, causal mechanisms can be identified throughout the phases of implementation. We use a hypothetical case study of a parenting program in the child welfare sector to illustrate the approach following a staged process analysis model taken from the econometric literature. Following the research on causal inference in recursive structural models, we also demonstrate how graphical models can be used to identify treatment effects at different levels of the system. Results: The models presented make contributions to implementation research in two areas. First, structural systems focus on the causal mechanisms of implementation. This allows efficient selection of covariates and optimal choice of implementation strategies based on identified leverage points in the system. Second, structural models provide a rigorous approach to the evaluation of implementation success. Taking a system approach, implementation, systems and effectiveness outcomes can be investigated as causally linked parts of an implementation system. Conclusions: As implementation science shifts its focus towards large dynamic systems of care, structural causal models connecting actors’ choices and implementation phases across levels of the system provide a rigorous scientific approach to investigate implementation and intervention effectiveness as causally linked parts of the system. Furthermore, using this approach it is straight forward to extend static models of implementation to dynamic systems that can account for sustainment and provide an avenue to integrate continuous quality improvement cycles into implementation research.