We Might Be Afraid Of Black-Box Algorithms

Fears of black-box algorithms are multiplying. They are said to prevent accountability,1 to make it harder to detect bias2 and so on. Some fears concern the epistemology of black-box algorithms in medicine and the ethical implications of that epistemology. In ‘Who is afraid of black box algorithms? On the epistemological and ethical basis of trust in medical AI,’3 Juan Duran and Jongsma seek to allay such fears. While we find some of their arguments compelling, we still see reasons for fear.\n\n### The gap between epistemic and normative justification\n\nDuranand Jongsma’s main claim is that black-box algorithms can confer epistemic justification.They helpfully note the scope of this claim’s implications: its truth would not alone give the ethical stamp of approval to decisions that are informed by black-box algorithms. For example, a clinician may be epistemically justified—on the basis of a black-box algorithm’s diagnosis—in believing that a patient has cancer without thereby being normatively justified in (say) administering chemotherapy. The epistemic justification can inform the decision, but further, moral considerations are needed to normatively justify it. This is a general point about the different natures of epistemic and normative justification.\n\nYet, especially in the medical context, the distinction between epistemic and normative considerations are not always clear-cut. For example, an algorithm that looks like it plays a …