Reconstructing Reality: From Physical World to Virtual Environments

With increasing availability of data in various forms from images, audio, video, 3D models, motion capture, simulation results, to satellite imagery, representative samples of the various dynamic phenomena constituting the physical world around us bring new opportunities and research challenges to Virtual Reality (VR). Such availability of data has also led to recent advances in data-driven modeling and data sciences. However, most of the existing example-based synthesis methods offer empirical models and data reconstruction that may not provide an insightful understanding of the underlying physical process or may be limited to a subset of observations. In this talk, I present recent advances that integrate classical model-based techniques and statistical methods to tackle challenging problems that have not been previously addressed in VR. These include flow reconstruction for traffic visualization, simultaneous estimation of physical deformation and elasticity parameters from images and video, and example-based multimodal display for VR systems. These approaches offer new insights for understanding complex collective behaviors, developing better models for complex dynamical systems from captured data, delivering more effective medical diagnosis and treatment, as well as cyber-manufacturing of customized apparel. I conclude by discussing some possible future directions and challenges.