Comparing evidence of selection bias between cluster-randomised and individually randomised controlled trials: a systematic review and meta-analysis

Abstract Background Cluster-randomised controlled trials require different methodology from individually randomised controlled trials and have unique vulnerabilities to bias. We aimed to compare selection bias between samples of these two types of trials published in four high-impact journals. Methods The JAMA archives and OVID Medline were searched by single selection by one of us (JB) for cluster-randomised and individually randomised controlled trials published in four journals ( BMJ, The Lancet, JAMA , and The New England Journal of Medicine). Two of us (JB and NM) independently double-extracted data from the 20 most recently published trials of each type up to July 3, 2017. Fixed-effects forest plots were generated to show any imbalances in baseline mean participant age between trial arms for each trial. Pooled imbalance was calculated for each trial type. The characteristic of age was chosen because it is reported universally, in standard units (years). Findings For individually randomised controlled trials, age imbalance between trial arms was not statistically significant (0·005 years, 95% CI −0·026 to 0·035). For cluster-randomised controlled trials, age imbalance was ten times greater and was statistically significant (−0·050, −0·057 to −0·043). Interpretation Randomisation distributes participant characteristics equally between trial arms except when baseline imbalances occur at random. When studies are pooled, such random imbalances cancel out to become negligible: if imbalance is not negligible across pooled trials, it indicates compromised randomisation or that selection bias has acted on the sample. The significant age imbalance in the cluster-randomised but not the individually randomised trials suggests that cluster-randomised trials are more vulnerable to selection bias. This imbalance might not affect trial outcomes since age in the control group was not substantially greater than in the intervention group. However, it indicates that selection bias can enter though the design of cluster-randomised controlled trials—possibly during the widespread practice of post-randomisation recruitment—and is therefore concerning. One limitation is that selection bias levels may be different in a more general sample of trials than one from four high-impact journals. Our method of examining selection bias has indicated that cluster-randomised controlled trials may require a more robust approach. We recommend taking steps to minimise selection bias in this type of trial. Funding None.