Be careful what you wish for: Commentary on Ebersole et al. (2016)

Ebersole et al.'s (2016) attempt to replicate Monin and Miller (2001) raises important questions about choosing beforehand which statistical test is the target of a replication. While our original theory a priori only predicted a main effect of the credentials manipulation, we had observed in the study reproduced here an unexpected interaction with participant gender. The current paper fails to replicate this originally unpredicted interaction, which it initially codes as a failure (Table 3), but significantly replicates the main effect of the credentials manipulation as an “added effect.” Ebersole et al. graciously acknowledge that the latter effect is of greater theoretical importance to our original theory, but the fact remains that the main pre-registered prediction for this replication (the interaction with gender) failed to garner empirical support. In this brief commentary I discuss issues with deciding a priori what should count as a replication, and own up to my responsibility making the ill-fated gender interaction be the centerpiece of this replication attempt.