Whose AI Dream? In search of the aspiration in data annotation

Abstract: This paper present the practice of data annotation from the perspective of the annotators. Data is fundamental to ML models. This paper investigates the work practices concerning data annotation as performed in the industry, in India. Previous investigations have largely focused on annotator subjectivity, bias and efficiency. We present a wider perspective of the data annotation, following a grounded approach, we conducted three sets of interviews with 25 annotators, 10 industry experts and 12 ML practitioners. Our results show that the work of annotators is dictated by the interests, priorities and values of others above their station. More than technical, we contend that data annotation is a systematic exercise of power through organizational structure and practice. We propose a set of implications for how we can cultivate and encourage better practice to balance the tension between the need for high quality data at low cost and the annotator aspiration for well being, career perspective, and active participation in building the AI dream.