The Reliability and Validity of Response-Based Measures of Attention Bias

Background Attentional bias to threat is a fundamental transdiagnostic component and potential vulnerability factor for internalizing psychopathologies. However, the measurement of attentional bias, such as traditional scores from the dot-probe paradigm, evidence poor reliability and do not measure intra-individual variation in attentional bias. Methods The present study examined, in three independent samples, the psychometric properties of a novel attentional bias (AB) scoring method of the dot-probe task based on responses to individual trials. For six AB scores derived using the response-based approach, we assessed the internal consistency, test–retest reliability, familial associations, and external validity (using Social Anxiety Disorder, a disorder strongly associated with attentional bias to threatening faces). Results Compared to traditional AB scores, response-based scores had generally better internal consistency (range of Cronbach’s alphas: 0.68–0.92 vs. 0.41–0.71), higher test–retest reliabilities (range of Pearson’s correlations: 0.26–0.77 vs. − 0.05 to 0.35), and were more strongly related in family members (range of ICCs: 0.11–0.27 vs. 0–0.05). Furthermore, three response-based scores added incremental validity beyond traditional scores and gender in the external validators of current and lifetime Social Anxiety Disorder. Conclusions Findings indicate that response-based AB scores from the dot-probe task have better psychometric properties than traditional scores.