Meta-analysis

Meta-analysis is the quantitative synthesis of the results of studies on a common topic or question. Its key goals are to assess the magnitudes of effects from the studies, and to quantify and explore variation in those effects. This chapter describes the set of practical and statistical procedures required to conduct a high-quality meta-analysis. The process begins with the formulation of a problem. This is followed by the data-collection step, during which the meta-analyst conducts a reproducible, exhaustive, and unbiased search for relevant studies that meet specific inclusion criteria. In the data-evaluation stage the meta-analyst extracts effect indices to represent study results, along with features of the participants, treatments (if any), methods, and measures used, and indicators of study quality. During data analysis, study features serve as potential explanations of between-studies differences. Finally, the review process and meta-analytic results are fully described and interpreted in the meta-analysis report. We illustrate these steps using two recent meta-analyses of studies of game-based instruction for second-language learning. Analyses based on data from Thompson and von Gillern (2020) illustrate fixed- and random-effects models for estimating overall effect magnitudes and their variance across studies, and for exploring predictors of between-studies differences. We also demonstrate the use of Bayesian meta-analysis to obtain information on the potential distributions of population parameters.