Solving the Black Box Problem: A Normative Framework for Explainable Artificial Intelligence

Abstract: Many of the computing systems programmed using Machine Learning are opaque: it is difficult to know why they do what they do or how they work. The Explainable Artificial Intelligence research program aims to develop analytic techniques with which to render opaque computing systems transparent, but lacks a normative framework with which to evaluate these techniques' explanatory success. The aim of the present discussion is to develop such a framework, while paying particular attention to different stakeholders' distinct explanatory requirements. Building on an analysis of 'opacity' from philosophy of science, this framework is modeled after David Marr's influential account of explanation in cognitive science. Thus, the framework distinguishes between the different questions that might be asked about an opaque computing system, and specifies the general way in which these questions should be answered. By applying this normative framework to current techniques such as input heatmapping, feature-detector identification, and diagnostic classification, it will be possible to determine whether and to what extent the Black Box Problem can be solved.