The PUEVA Inventory: A Toolkit to Evaluate the Personality, Usability and Enjoyability of Voice Agents

Abstract: The proliferation of voice agents in consumer devices requires new tools for evaluating these systems beyond their technical functionality. This paper presents a toolkit for the evaluation of Voice User Interfaces (VUIs) with the intention of measuring the crucial factors of subjective enjoyment in the user experience. The PUEVA toolkit was constructed using a meta-analysis of existing literature, structured N=20 and semi-structured N=18 interviews and a within subjects lab study. The resulting questionnaire contains 35 items that represent 12 scales in three categories: (1) Personality (2) Usablity and (3) Enjoyability. The PUEVA Toolkit moves us towards the capacity to evaluate and compare subjective, joyful experiences in between-subject as well as within-subject research designs.