Now Is the Time to Assess the Effects of Open Science PracticesWith Randomized Control Trials

Public Significance Statement We call for experimental trials to test the effects of Open Science research practices, an initiative that could improve the entire scientific enterprise. Well-designed and implemented randomized controlled trials can assist in specifying the effects of different practices and inform decisions regarding their implementation. We issue a call for the design and conduct of experimental trials to test the effects of researchers' adoption of Open Science (OS) research practices. OS emerged to address lapses in the transparency, quality, integrity, and reproducibility of research by proposing that investigators institute practices, such as preregistering study hypotheses, procedures, and statistical analyses, before launching their research. These practices have been greeted with enthusiasm by some parts of the scientific community, but empirical evidence of their effects relies mainly on observational studies; furthermore, questions remain about the time and effort required by these practices and their ultimate benefit to science. To assess the outcomes of OS research practices, we propose they be viewed as behavioral interventions for scientists and tested in randomized controlled trials (RCTs), to identify potential benefits and (unintended) harms. As this is a call to action rather than an action plan per se, we sketch out four potential trial designs to encourage further deliberation and planning. Experimental tests to document the outcomes of OS practices can provide evidence to optimize how scientists, funders, policymakers, and institutions utilize these strategies to advance scientific practice.