No Evidence of Bias When Using Inappropriate Test for Bias: Comment on Cesario, Johnson, & Terrill 2018

A recently published paper claimed that there is no evidence of anti-black bias in police shootings (Cesario, Johnson & Terrill 2018). They compare the ratio of blacks and whites shot by police to the number of violent crimes committed by each race, instead of to population levels. Based on this they find no evidence for anti-black bias in police shootings, actually finding an anti-white bias in many cases. However, the methods used by the authors are unable to determine if bias exists. We illustrate by showing that their method is unable to distinguish between a hypothetical world in which no bias exists, and one in which severe anti-black bias exists. Analysis of the same shootings database used by the authors further indicate the authors’ assumptions do not match with the real world. Given this, any conclusions being made from the data and analysis used in the paper are unwarranted.