Clarifying the relationship between mental illness and recidivism using machine learning: A retrospective study

Objective There is currently inconclusive evidence regarding the relationship between recidivism and mental illness. This retrospective study aimed to use rigorous machine learning methods to understand the unique predictive utility of mental illness for recidivism in a general population (i.e.; not only those with mental illness) prison sample in the United States.Method Participants were adult men (n = 322) and women (n = 72) who were recruited from three prisons in the Midwest region of the United States. Three model comparisons using Bayesian correlated t-tests were conducted to understand the incremental predictive utility of mental illness, substance use, and crime and demographic variables for recidivism prediction. Three classification statistical algorithms were considered while evaluating model configurations for the t-tests: elastic net logistic regression (GLMnet), k-nearest neighbors (KNN), and random forests (RF).Results Rates of substance use disorders were particularly high in our sample (86.29%). Mental illness variables and substance use variables did not add predictive utility for recidivism prediction over and above crime and demographic variables. Exploratory analyses comparing the crime and demographic, substance use, and mental illness feature sets to null models found that only the crime and demographics model had an increased likelihood of improving recidivism prediction accuracy.Conclusions Despite not finding a direct relationship between mental illness and recidivism, treatment of mental illness in incarcerated populations is still essential due to the high rates of mental illnesses, the legal imperative, the possibility of decreasing institutional disciplinary burden, the opportunity to increase the effectiveness of rehabilitation programs in prison, and the potential to improve meaningful outcomes beyond recidivism following release.