Uncertainty in Criminal Justice Algorithms: simulation studies of the Pennsylvania Additive Classification Tool

Abstract: Much attention has been paid to algorithms related to sentencing, the setting of bail, parole decisions and recidivism while less attention has been paid to carceral algorithms, those algorithms used to determine an incarcerated individual's lived experience. In this paper we study one such algorithm, the Pennsylvania Additive Classification Tool (PACT) that assigns custody levels to incarcerated individuals. We analyze the PACT in ways that criminal justice algorithms are often analyzed: namely, we train an accurate machine learning model for the PACT; we study its fairness across sex, age and race; and we determine which features are most important. In addition to these conventional computations, we propose and carry out some new ways to study such algorithms. Instead of focusing on the outcomes themselves, we propose shifting our attention to the variability in the outcomes, especially because many carceral algorithms are used repeatedly and there can be a propagation of uncertainty. By carrying out several simulations of assigning custody levels, we shine light on problematic aspects of tools like the PACT.