Accounts, Accountability and Agency for Safe and Ethical AI

Abstract: We examine the problem of explainable AI (xAI) and explore what delivering xAI means in practice, particularly in contexts that involve formal or informal and ad-hoc collaboration where agency and accountability in decision-making are achieved and sustained interactionally. We use an example from an earlier study of collaborative decision-making in screening mammography and the difficulties users faced when trying to interpret the behavior of an AI tool to illustrate the challenges of delivering usable and effective xAI. We conclude by setting out a study programme for future research to help advance our understanding of xAI requirements for safe and ethical AI.