#### Department of Computer Science College of Computer, Mathematical, and Natural Sciences The University of Maryland at College Park

CMSC 828X (Fall 2022): Learning-based Modeling, Simulation and Animation

# CMSC 828X (Fall 2022): Learning-based Modeling, Simulation and Animation

## Instructor: Ming C. Lin

Time and Place: Tues/Thur 12:30pm - 1:45pm, CSIC 2120
Office Hours: Tues/Thur After Class
Prerequisites: Scientific Computing AND Introductory Computer Graphics (Optional) OR Instructor's approval
Textbook: Course Notes and In-Class Handouts

## Check out some cool videos or find them at YouTube!

• Course Overview
• Lectures and Approximate Schedule
• Assignments and Projects
• WEBS of Physically-Based Modeling & Animation
• Geometric Algorithms & Software Available on the Web
• Conferences in Related Areas
• ### COURSE OVERVIEW:

Physically-based modeling and simulation attempts to map a natural phenomena to a computer simulation program. There are two basic processes in this mapping: mathematical modeling and numerical solution. The goal of this introductory course is to understand both of them. The mathematical modeling concerns the description of natural phenomena by mathematical equations. Differential equations that govern dynamics and geometric representation of objects are typical ingredients of the mathematical model. The numerical solution involves computing an efficient and accurate solution of the mathematical equations. Finite precision of numbers, limited computational power and memory forces us to approximate the mathematical model with simple procedures.

In the first part of this course, we will study various techniques to simulate the physical and mechanical behavior of objects in a graphical simulation or a virtual environment. Students will learn about implementation of basic simulation programs that produce interesting results and verify its correctness. The course will cover three basic components in physically-based modeling and simulation:

• Geometry
• Collision Detection
• Computing Contact Manifolds
• Mechanics
• Particle Dynamics
• Rigid Body Dynamics
• Non-Rigid Body Dynamics
• Numerical Computing
• Initial Value Problems
• Boundary Value Problems
• Constraints & Differential-Algebraic Equations
• In the second part of the course, we'll study most recent advances on learning-based algorithms, built on the state-of-the-art physics-based modeling, simulations, and animation data to learn the dynamics and motion of moving objects and characters. The goal of this class is to get students an appreciation of computational methods for modeling of motions in the physical and virtual world through BOTH model-based and DATA-DRIVEN methods. We will discuss various considerations and tradeoffs used in designing simulation methodologies (e.g. time, space, robustness, and generality). This will include network models and architectures, data structures, algorithms, computational methods and learning-based motion reconstruction techniques, their complexity and implementation. The lectures will also cover some applications of physics vs. learning-based modeling and simulation to the following areas:
• Computer Animation
• Virtual Environments
• Medical Simulation and Analysis
• Special Effects Generation
• Computer Game Dynamics
• Rapid Prototyping for Design
• Haptic Rendering/Interfaces
• Robotics and Automation
• Bio-informatics
• Depending on the interests of the students, we may also cover geometric-based simulation techniques, such as constraint-based systems, optimization-based animation, inverse dynamics, kinematics of motions, motion planning and synethesis of autonomous agents.

### LECTURES AND APPROXIMATE SCHEDULE

Here is a list of TENTATIVE lecture topics (subject to changes). Schedule and information on each topic (e.g. readings, web pointers) will be added during the semester before each class.

• Overview (Tues, Aug 30, 2022)
• Basics of Motion Generations for Animation (Thur, Sept 1, 2022)
• Principles of Animation and Inverse Kinematics (Tues, Sept 6, 2022) with Supplementary Video
• ODE Basics: Initial Value Problem & (Thur, Sept 8, 2022)
• No Class (Tues, Sept 13, 2022) - Sub Live Demo Sesssion on Sept. 22
• Intro to Particle Dynamics & Particle Systems (Thur, Sept 15, 2022)
• Implicit Methods & Particle LODs & Solutions to Spring-Mass System (Tues, Sept 20, 2022)
• Constrained Dynamics (Thur, Sept. 22, 2022)
• Homework #1 Live Demo @ IRB 5107 (4:45pm-5:45pm) (Thur, Sept. 22, 2022)
• Review on Comp Geom (Tues, Sept 27, 2022)
• Review on Comp Geom & Collision Detection: Convex Polyhedra & Many Bodies (Thur, Sept 29, 2022)
• Collision Detection: Convex Polyhedra & Many Bodies - Makeup Lecture posted on ELMS (Tues, Oct 4, 2022)
• Collision Detection: BVHs & Spatial Partitioning - Makeup Lecture posted on ELMS (Tues/Thur, Oct 4-6, 2022)
• Project Proposal Presentations (Tues/Thur, Oct 11-13, 2022)
• Rigid Body Dynamics (Notes on Quaternion, Particle to Rigid Body Dynamics, and More) (Tues, Oct 18, 2022)
• Modeling of Deformable Models: Introduction (Thur, Oct 20, 2022)
• Machine Learning for Virtual Try-On (Tues, Oct 25, 2022)
• Differentiable Physics: Rigid, Articulated, Deformable & Cloth Simulation (Thurs, Oct 27, 2022)
• Recent Advances in Deformable Models and Hyperelastic Material Models (Tues, Nov 1, 2022)
• Introduction to Fluid Simulation and Differentiable Fluid Dynamics (Thur, Nov 3, 2022)
• Voronoi Diagrams and Applications in Motion Planning (Tues, Nov 8, 2022)
• Embodied AI and Indoor Navigation Simulators (Tues, Nov 8, 2022)
• Interactive Household Environment Simulator for Embodied AI (Nov 10, 2022)
• Digital Humans (Nov 10, 2022)
• Midterm Project Proress Report (Nov 15-17, 2022)
• Learning-based Simulation and Animation Using GNNs (Nov 17, 2022)
• Crowd & Traffic Simulations (Nov 22, 2022)
• THANKSGIVING BREAK (Nov 23-25, 2022)
• Reinforcement Learning for Dynamics Simulation (Nov 17/29, 2022)
• Hybrid Contact Modeling for Contact-rich Robot Manipulation (Nov 29, 2022)
• Motion Retargeting (Dec 1, 2022)
• N-body and Hydrodynamic Simulations in Astrophysics (Dec 1, 2022)
• AI in Real-time Interactive Simulations for Video Games (Dec 6, 2022)
• Traffic Modeling & Simulation (Dec 6, 2022)
• Brain-Computer Interfaces (Dec 8, 2022)
• Modeling Emotions Using EEG Signals (I) (Dec 8, 2022)

### Reference Papers Used in Lectures:

• SIGGRAPH Course Notes on Physically-Based Modeling
• Supplementary Materials and Reading List for the Class (updated throughout the semester)
• Additional Notes on Rigid Body Dynamics: Constraint-based Dynamics, and Impulse-based Dynamics

• ### ASSIGNMENTS AND PROJECTS

The class grade of each student is determined by
• Homework (30%)
• Class Presentation (20%)
• Final Project (50%)

### SELECTED INDUSTRY

• Boston Dynamics Inc.
• Chris Hecker's Corner (Definition Six, Inc.)
• Dreamworks Animation
• GamaNetwork
• Havok
• IBM Smarter Planet
• Immersion Corporation
• Legion Limited
• Massive Software
• MAYA (Alias|Wavefront)
• MSC.Working Knowledge
• NVIDIA PHYSX
• Pixar Animation Studios
• Rhythm & Hues Studios
• SensAble Technology
• Walt Disney Animation Studio
• Weta Digital
• ### SELECTED RESEARCH GROUPS:

• UNC Research Group on Geometric Algorithms for Modeling, Motion and Animation
• UNC Interactive Collision Detection and Proximity Queries Packages
• Simlab: Computer Tools for Analysis and Simulation (Cornell)
• iMAGIS (GRAVIR / IMAG research lab / INRIA)
• Center for Human Modeling and Simulation(UPENN)
• MIRALab (University of Geneva)
• Rutgers Computational Biomedicine Imaging and Modeling (Rutgers)
• University of Aukland, Bioengineering Research Group
• ### SELECTED RESEARCHERS

• David Baraff (now at Pixar)
• Ronan Boulic
• David Breen
• Chris Bregler
• Robert Bridson
• Marie-Paule Cani
• Stephen Chenney
• Mathieu Desbrun
• Petros Faloutsos
• Ron Fedkiw
• Eugene Fiume
• Doug James
• Jessica Hodgins
• Michael Gleicher
• Hyeong-Seok Ko
• Dimitris Metaxas
• Brian Mirtich (now at Cognex)
• James O'Brien
• Carol O'Sullivan
• Richard Parent
• Dinesh Pai
• Nancy Pollard
• Jovan Popovic
• Zoran Popovic
• Jos Stam
• Peter Schroder
• Karan Singh
• Daniel Thalmann
• Demetri Terzopoulos
• Yizhou Yu
• Michiel van de Panne
• Andy Witkin's Gallery (now at Pixar)
• ### GEOMETRIC ALGORITHMS AND SOFTWARES AVAILABLE ON THE WEB:

Here are just some possible locations to find geometric software/libraries and algorithmic toolkits you may need:
• Internet Finite Element Resources
• A comprehensive collection of geometric software
• CGAL: Computational Geometry Algorithms Library (in C++)
• LEDA: Library of Efficient Datatypes and Algorithms (in C++)
• The Stony Brook Algorithm Repository: Implementation in C, C++, Pascal and Fortran
• CMU's Computer Vision Homepage
• Finite element mesh generation and More
• Machine learning resources

### Other Reference Books in Computer Animation:

• Making Them Move: Mechanics, Control and Animation of Articulated Figures, by Badler, Barsky and Zelter, Morgan Kaufmann Publishers, 1991.
• Advanced Animation and Rendering Techniques: Theory and Practice, by A. Watt and M. Watt, 1992.
• Computer Animation: Algorithms and Techniques, by Rick Parent, 1999.
• ### Other Reference Books in Mechanics:

• Concepts and Applications of Finite Element Analysis, by R. D. Cook, D. S. Malkus and M. E. Plesha, John Wiley & Sons, 1989.
• Finite Element Procedures, by K.-J. Bathe, Prentice Hall, 1996.
• First Course in Continuum Mechanics, by Y.C. Fung, Prentice Hall, 1993.
• ### Other Reference Books in Numerical Methods:

• Numerical Recipes, by Press, Flanner, Teukolsky and Vetterling, Cambridge University Press.
• Handbook of Numerical Analysis, edited by Ciarlet and Lions, Vol. I - VI, North-Holland, 1994.
• ### Other Reference Books in Robotics:

• Robot Motion Planning, by Latombe, Kluwer Academic Publishers, 1991.
• Planning Algorithms, by LaValle, Cambridge University Press, 2006.
• Robot Manipulators: Mathematics, Programming, and Control, by R. P. Paul, MIT Press, 1981.
• ### Other Reference Books in Geometry:

• Computational Geometry (Algorithms and Applications), by de Berg, van Kreveld, Overmars and Schwarzkofp, Springer-Verlag, 1997.
• Computational Geometry In C (Second Edition), by O'Rourke, Cambridge University Press, 1998.
• Handbook on Discrete and Computational Geometry, by Goodman and O'Rourke (eds), CRC Press LLC, 1997.
• Applied Computational Geometry: Toward Geometric Engineering, by Lin and Manocha (eds), Springer-Verlag, 1996.
• Algorithms in Combinatorial Geometry, by Edelsbrunner, Springer-Verlag, 1987.
• Computational Geometry (An Introduction), by Preparata and Shamos, Springer-Verlag, 1985.