Heterogeneity in Mathematics Intervention Effects: Evidence from a Meta-Analysis of 191 Randomized Experiments

Since the standards-based education movement began in the early 1990s, mathematics education reformers have developed and evaluated many interventions to support students in mastering more rigorous content. We conducted a systematic review and meta-analysis of U.S. PreK-12 mathematics intervention effects from 1991 to 2017 to study sources of heterogeneity. From more than 9,000 published and unpublished study reports, we found 191 randomized control trials that met our inclusion criteria, with 1,109 effect size estimates representing more than a quarter of a million students. The average effect size on student mathematics achievement was 0.31, with wide heterogeneity of most effects ranging from -0.60 to 1.23. Two modeling approaches-meta-regression and machine learning-provided converging evidence that outcome measure type (researcher-created vs. standardized) and technology delivery (vs. teacher or interventionist delivery) were predictors of effect size. Intervention type, intervention length, grade level, and publication year were also identified as potentially explanatory factors.