PhD Proposal: Acoustic Simulation for Learning-based Virtual and Real World Applications

Sound propagation is commonly perceived as air pressure perturbations due to vibrating/moving objects. The energy of sound gets attenuated by transmitting in the air over a distance, and by being absorbed at other object surfaces. Numerous research have focused on devising better acoustic simulation methods to model sound propagation in a more realistic manner. The benefits of accurate acoustic simulations include but are not limited to: computer-aided acoustic design, acoustic optimization, synthetic speech data generation, and immersive audio-visual rendering for mixed reality.One standing problem in the acoustic simulation field is the tradeoff between accuracy and time-space cost. Conventional numeric wave solvers based on the first-principal wave equation provides the most accurate results that can be validated with real-world measurements. However, they usually scale poorly with simulation frequency and scene scale, making them unsuitable for large simulations (in amount or scale). In recent years, more efficient geometric acoustic simulators have been developed that rely on the assumption that have high frequency sound travels like rays despite their wave intrinsic. While being way more efficient than wave based solvers, the most widely used geometric simulator (e.g., the image method) has drawbacks that cause a big gap between its output and real-world measurements. Therefore, our first goal is to devise better acoustic simulation methods that can capture the missing components of the image method and generate better simulated data, which can potentially improve the performance of data-driven applications using synthetic acoustic data.Another challenge is how to incorporate good synthetic sound in real-world settings, where the simulated sound needs to be consistent with the recorded sound. This requires our simulation to be scene-aware: the sound simulation setups need to align with the real-world scene. The main difficulty comes from two parts: 1) The real-world scene configurations are not always well known, which need to be empirically inferred or measured on-site. Prior solutions are either inaccurate or un-user-friendly. 2) The wave effects are essential for low-frequency components, but are poorly approximated by state-of-the-art real-time geometric acoustic simulators. A large amount of pre-computation time is needed to incorporate results from wave based solvers. To address these problems, we seek novel solutions via acoustic simulation and deep learning to provide high quality sound rendering in mixed reality settings while having fewer limitations than existing methods.Examining Committee:

Chair: Dr. Dinesh Manocha Dept rep: Dr. Nirupam Roy Members: Dr. Ramani Duraiswami Dr. Ming Lin