PhD Proposal: Learning-Based Physics Simulation with Collision Handling

Physics-based simulation and collision handling are fundamental components of different applications such as animation, virtual try-on systems, immersive audio-visual interactions, and multi-robot navigation. Physics-based simulation systems need to account for constitutive models of materials and many hard constraints, including collision-free simulations.An ideal balance between the accuracy and efficacy of simulation still faces several challenges. First, the computational cost grows superlinearly with the dimension of the physical system’s configuration space. For example, a high-resolution cloth used in animation or games typically involves tens of thousands of degrees of freedom. As another example, to simulate cloth or body tissues with vivid details, the finite element method involves solving large global systems of equations. Second, it is generally difficult to acquire accurate and complete information or material properties from the real world. Recent advances in machine learning have shown potential in dealing with many of these challenges. Deep learning models can capture details through learned filters and compress high-dimensional data into low-dimensional latent vectors using autoencoders. Furthermore, they can provide real-time performance on commodity GPUs for complex benchmarks.Inspired by these advances, we present novel learning-based physics simulations and collision handling algorithms. Our work is motivated by the following applications: collision-free cloth neural simulation and collision response for human bodies or general 3D model deformation. Under these applications, we inject domain knowledge, i.e., physical models and collision-free constraints, into deep learning architectures. Our work addresses three major issues:

Physical-inspired loss: We train real-time, learning-based physics simulators using physical- inspired loss terms and consider kinetic and potential energies. Our embedding approach leads to better accuracy in adhering to the laws of physics with up to 70% improvement. We observe 500-10000X speedups over physics-based simulators running in high-dimensional configuration spaces.
Learned constraints for fast collision handling: We train a learning-based collision detector for 3D human models and use the detector as a surrogate constraint in an optimization-based collision handler. Based on these techniques, we achieve an accuracy of 94.1% when predicting collisions for randomized human poses sampled from widely-used datasets. After learning the feasible domain, solving a constrained optimization for a collision-free human pose with thousands of vertices takes a fraction of a second.
Inference-phase collision response based on a learned gradient: We present an approach to speed up collision response computations by introducing an additional repulsive force unit in the learning-based pipeline. Our approach approximates the distance field and penetration energy for self- or inter-object collisions. In addition, we also predict the gradient and a suitable step length for reducing penetration. Our experiments show that backbone networks trained with the repulsive force unit can significantly reduce the number of collisions while maintaining real-time performance.

Examining Committee:

Chair:Department Representative:

Dr. Dinesh Manocha Dr. Matthias ZwickerDr. Ming Lin