Users’ needs in interactive bias auditing tools introducing a requirement checklist and evaluating existing tools

In the past, automated decision-making (ADM) models have been shown to adopt biases from the data they have been trained on and make discriminatory decisions based on individuals’ gender, age, race and intersections of these. To make sure that these unwanted biases are found before models are deployed, interactive auditing tools, that do not require programming knowledge of their users, have been developed. Since discriminatory patterns are typically quite subtle and may unfold in complex ways, such tools need to offer a wide range of functionalities, to make sure that auditors can detect, understand, and contextualize all the important biases within a model. Many interviews and usability studies have been conducted to identify the functional requirements an optimal tool should satisfy. Yet, there exists no extensive checklist of these requirements, nor is it clear to which extent current auditing tools fulfil them. In this paper, we are the first to provide an overview of currently existing tools, while also encapsulating auditors’ functional needs for such tools in one comprehensive checklist. More importantly, we will evaluate each of the existing tools according to this checklist and identify ways their shortcomings can be overcome. Common points of improvement we identified using our checklist, concern the tools’ functionality to let users detect complex forms of bias (like intersectional bias) and let users understand the causes of this bias.