A Qualitative Post-Experience Method for Evaluating Changes in VR Presence Experience Over Time

Abstract: A particular measure to evaluate a head-mounted display (HMD) based experience is the state of feeling present in virtual reality. Interruptions of a presence experience - break in presence (BIP) - appearing over time, need to be detected to assess and improve an application. Existing methods either lack in taking these BIPs into account - questionnaires - or are complex in their application and evaluation - physiological and behavioral measures -. To provide a practical approach, we propose a post-experience method in which the users reflect on their experience by drawing a line, indicating their experienced state of presence, in a paper-based drawing template. The amplitude of the drawn line represents the variation of their presence experience over time. We propose a descriptive model that describes temporal variations in the drawings by the definition of relevant points over time - e.g., putting on the HMD -, phases of the experience - e.g., transition into VR - and parameters - e.g., the transition time -. The descriptive model enables us to objectively evaluate user drawings and represent the course of the drawings by a defined set of parameters. An exploratory user study (N=30) showed that the drawings are very consistent, the method can detect all BIPs and shows good indications for representing the intensity of a BIP. With our method practitioners and researchers can accelerate the evaluation and optimization of experiences by evaluating BIPs. The possibility to store objective parameters paves the way for automated evaluation methods and big data approaches.