Towards Modeling Software Quality of Virtual Reality Applications from Users' Perspectives

Abstract: Virtual Reality (VR) technology has become increasingly popular in recent years as a key enabler of the Metaverse. VR applications have unique characteristics, including the revolutionized human-computer interaction mechanisms, that distinguish them from traditional software. Hence, user expectations for the software quality of VR applications diverge from those for traditional software. Investigating these quality expectations is crucial for the effective development and maintenance of VR applications, which remains an under-explored area in prior research. To bridge the gap, we conduct the first large-scale empirical study to model the software quality of VR applications from users' perspectives. To this end, we analyze 1,132,056 user reviews of 14,150 VR applications across seven app stores through a semiautomatic review mining approach. We construct a taxonomy of 12 software quality attributes that are of major concern to VR users. Our analysis reveals that the VR-specific quality attributes are of utmost importance to users, which are closely related to the most unique properties of VR applications like revolutionized interaction mechanisms and immersive experiences. Our examination of relevant user complaints reveals the major factors impacting user satisfaction with VR-specific quality attributes. We identify that poor design or implementation of the movement mechanisms, control mechanisms, multimedia systems, and physics, can significantly degrade the user experience. Moreover, we discuss the implications of VR quality assurance for both developers and researchers to shed light on future work. For instance, we suggest developers implement sufficient accessibility and comfort options for users with mobility limitations, sensory impairments, and other specific needs to customize the interaction mechanisms. Our datasets and results will be released to facilitate follow-up studies.