Developing effective virtual simulations and serious games: the effect of background sound cues on visual quality perception

Abstract

Virtual simulations and serious games (video game-based technologies applied to teaching and learning) have been incorporated in the teaching and training curricula of a large number of professions including medicine/surgery. Despite their benefits, there are open, fundamental issues regarding simulation quality, multi-modal cue interaction, and the resulting effect on visual quality perception and ultimately on knowledge transfer and retention. Here the results of a series of seven studies that examined the effect of background sound (contextually related and non-related with respect to the visual scene) on the perception of visual quality (defined with respect to texture resolution, polygon count) presented in stereoscopic and non-stereoscopic 3D. Results indicate that the perception of visual quality is dependent on ambient (background) sound. The results of these studies have implications for designers and developers of serious games who typically strive for high quality virtual worlds despite the computational burden associated with doing so. The results of these studies also bring us closer to understanding the role of quality, multi-modal interactions, and their effect on visual quality perception. This thesis is part of a larger effort in developing an understanding of virtual environment rendering quality, multi-modal interactions, user-specific factors and their effect on knowledge transfer and retention.