IEEE Digital Reality: Learning and Training in VR: Hype and New Hope

The ability for Virtual Reality (VR) to create rich and compelling experiences drives research in a variety of domains beyond entertainment. One such area of interest is the use of VR for learning and training purposes. Indeed, learning and training have been a central topic in VR ever since successful flight simulators decades ago. The allure of VR for training is not only due to the ability to synthesize realistic conditions for situations where on-site training is expensive, dangerous, or difficult to replicate in real life, but also due to the ability to create extreme, unrealistic conditions to over-prepare trainees for the real situation. Full control over the visual and auditory experience of the user, as well as some control over haptic feedback, makes for an engaging training experience and ensures the reproducibility of training sessions. Combined with recent advances in light and untethered headsets, tracking, interaction, and VR authoring tools, various industrial immersive learning applications show promising results. The explosive potential of VR-based collaboration in the post-pandemic era, where remote and non-contact work environments became the new normal, further accelerated the adoption of VR-based training in broader use cases.

Despite the large media coverage of VR training stories for various industrial applications, there are still mixed scientific reports on the efficacy and measurable performance improvements depending on significant factors such as the domain of training, implementation, level of engagement and interaction, and the complexity of skills and tasks. In this talk, Professor Choi presents the taxonomy of learning and training in VR/AR/MR and discusses the status of technological advances and limitations. Beyond visual and auditory-focused immersion, kinesthetic learning for fine motor skills poses a new challenge for VR-based training. Developing the ability to perform a new skill, and retaining that ability, requires repetitive coordination of the brain and muscles (sports training in VR is a prime example). Can the complex motor skills required in sports be learned in VR? If so, do these skills transfer to the real world? Can it provide a lasting impact similar to real-world training? This talk also addresses core endeavors focused on feedback and adaptive training mechanisms.