The Nature of ROC Practices in Eyewitness Memory Research

Eyewitness memory research has reformed police practices and policy and is sometimes relied upon in legal proceedings. Due to the practical implications derived from this research, it is imperative to evaluate how practical recommendations are postulated. To assess the practical relevance of research, effect sizes and its interpretation play a pivotal role. In this study, we examined how the frequently used effect size Area Under the Curve (AUC) obtained via Receiver Operating Characteristic (ROC) curves are used and interpreted in eyewitness memory research. Via a systematic literature search, we identified 157 eyewitness memory related articles that conducted ROC curve analyses resulting in 1580 AUC values. Approximately 90% of the studies relied on statistical significance to interpret the AUC values. This practice can be problematic as the sample sizes in many studies were rather large which can lead to statistically significant effects that are trivial. Also, more than half of the studies did not report 95% CIs for their AUC values. Finally, power analyses were frequently not conducted and if they were they were, they were oftentimes not reproducible. To improve the practical inferences of eyewitness memory research, we highlight the need of establishing a smallest effect size of interest, focusing on 95% CIs, and conducting reproducible power analyses.