Running Head: WEAK AND VARIABLE EFFECTS OF TESTOSTERONE ON CRT Weak and variable effects of exogenous testosterone on cognitive reflection task performance in three experiments: Commentary on Nave et al. (2017)

Nave and colleagues (2017) presented a single experiment (n=243) finding that exogenous testosterone caused a decrease in performance on the Cognitive Reflection Test (CRT) by increasing intuitive-but-incorrect responses. We report three new experiments (total n=628) that also examine the effect of exogenous testosterone on CRT performance. When pooling the data across experiments, we find (i) substantial variation in CRT performance across experiments, treatment groups, and participants and (ii) variable treatment effects of testosterone on CRT performance across experiments with any average effect being weak relative to this underlying variability – regardless of whether we considered the three new experiments or all four. Given our modeling assumptions, an average treatment effect of a 7% decrease in the odds of correctly responding to a CRT item is the value most compatible with the data from the three new experiments; however, anything from a 53% decrease to a 99% increase is also reasonably compatible. Similarly, a 27% decrease is the value most compatible with the data from all four experiments; however, anything from a 62% decrease to a 58% increase is also reasonably compatible. We explore potential explanations for the pattern of results observed across the four experiments.