Performance-Guided Synthesis of Personalized Virtual Environments for the Future of Work & Training

GMU Professor Lap-Fai (Craig) Yu delivers a clear vision for how virtual and augmented reality - aided by machine learning - will enter into our everyday work and home lives.

Dr. Yu discusses how automatically synthesized virtual training content can unlock the potential of virtual reality for a number of compelling and novel training applications. In particular, he discusses how to apply optimization and machine learning approaches to generate a large variety of virtual environments, from apartments, restaurants, factories, train stations, to shopping malls.

Paired with a trainee's performance data such as perception and behavior tracked in VR, these approaches can also incorporate personal factors including skill levels, weaknesses, preferences, and training goals for synthesizing personalized virtual training experiences. One can also realize smart interfaces such as adaptive VR/AR navigation aids driven by gaze.

Dr. Yu illustrates the approaches with a number of training examples such as earthquake safety training, driving training, and body fitness training in virtual reality.