Can Multisensory Cues in VR Help Train Pattern Recognition to Citizen Scientists?

Abstract: As the internet of things (IoT) has integrated physical and digital technologies, designing for multiple sensory media (mulsemedia) has become more attainable. Designing technology for multiple senses has the capacity to improve virtual realism, extend our ability to process information, and more easily transfer knowledge between physical and digital environments. HCI researchers are beginning to explore the viability of integrating multimedia into virtual experiences, however research has yet to consider whether mulsemedia truly enhances realism, immersion and knowledge transfer. My work developing StreamBED, a VR training platform to train citizen science water monitors plans to consider the role of mulsemedia in immersion and learning goals. Future findings about the role of mulsemedia in learning contexts will potentially allow learners to experience, connect to, learn from spaces that are impossible to experience firsthand.