Systematic variability enhances the reproducibility of an ecological study

Many scientific disciplines currently are experiencing a reproducibility crisis because numerous scientific findings cannot be repeated consistently. A new but controversial hypothesis postulates that stringent levels of environmental and biotic standardization in experimental studies reduces reproducibility by amplifying impacts of lab-specific environmental factors not accounted for in study designs. A corollary to this hypothesis is that the deliberate introduction of controlled systematic variability (CSV) in experimental designs can increase reproducibility. We tested this hypothesis using a multi-laboratory microcosm study in which the same ecological experiment was repeated in 14 laboratories. Each laboratory introduced environmental and genotypic CSV within and among treatments in replicated microcosms established in either growth chambers (with stringent control of environmental conditions) or glasshouses (with more variable environmental conditions). The introduction of genotypic CSV increased reproducibility of results in growth chambers but had no significant effect in glasshouses where reproducibility also was lower. Environmental CSV had little effect on reproducibility. This first deliberate attempt at reproducing an ecological experiment with added CSV reveals that introducing genotypic CSV in experiments carried out under controlled environmental conditions with stringent standardization can increase reproducibility by buffering against unaccounted lab-specific environmental and biotic factors that may otherwise strongly bias experimental outcomes.